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ABSTRACT 

The  Naval  Underwater  Systems  Center  (NUSC)  located  in 
New  London,  Connecticut,  has  a  need  for  an  Information  Flow 
and  Analysis  System  (IFAS)  for  the  Sonar  Operational  Training 
and  Assessment  Program  (SOTAP) .   The  study  addresses  the 
requirements  for  sonar  operational  programs.   It  discusses 
basic  differences  between  weapons  and  information  systems, 
and  proposes  a  systems  approach  for  the  acquisition  of  a 
basic  management  information  system.   It  presents  the  infor- 
mation system  alternatives  available  to  the  SOTAP  management 
and  describes  the  existing  information  system,  Personnel 
Training  and  Evaluation  Program  (PTEP) .   It  disucsses  PTEP ' s 
FY  78  incorporation  into  the  Navy's  Versatile  Training  System 
(VTS)  and  how  PTEP  may  be  expanded  and  changed  under  VTS  to 
include  the  sonar  rating  aboard  Fleet  Ballistic  Missile  (FBM) 
submarines  with  the  end  goal  of  improving  ultimate  user  (sonar 
technicians)  knowledge  and  performance  of  sonar  weapon 
systems . 
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I.   INTRODUCTION 

Sonar,  an  acronym  for  SOund  NAvigation  and  Ranging, 
designates  that  branch  of  applied  acoustics  in  which  acous- 
tic energy  is  propagated  through  a  water  medium  [Ref.  1] . 
Systems  which  utilize  underwater  acoustic  energy  for  obser- 
vation or  communications  are  referred  to  as  sonar  systems. 
They  are  used  for  many  purposes  ranging  from  peaceful  "fish 
finders"  and  small  boat  navigation  aids  to  large  anti- 
submarine warfare  (ASW)  systems  for  detection  and  classi- 
fication of  ships,  submarines,  and  mine  hunting.   Sonar 
systems  also  provide  a  means  for  both  short  and  long 
distance  underwater  communications. 

A.   BACKGROUND  OF  SOTAP 

A  sonar  on-board  trainer  (OBT)  is  believed  by  Mr.  Russell 
L.  Brown,  Principal  Investigator  (SOTAP)  to  be  needed  for 
submarines  but  acquisition  attempts  until  recently  have  not 
been  fruitful.   In  the  spring  of  1973  a  sonar  on-board 
trainer  was  sea  tested  on  a  non-Digital  Multi-Beam  Steering 
(DIMUS)  sonar  suite  aboard  the  U.S.S.  WILLIAM  H.  BATES 
(SSN-680).   An  OBT  is  an  Advanced  Development  Model  (ADM) 
piece  of  hardware  that  can  inject  realistic  target  signals 
into  the  sonar  suite.   During  the  sea  trials  test  of  the 
hardware,  the  question  of  "How  were  the  ship's  personnel 
going  to  use  the  OBT?"  became  apparent  to  the  sea  trials 


test  director,  Mr.  Russell  L.  Brown.   By  the  end  of  the  sea 
trials  it  was  concluded  that  this  piece  of  hardware  would 
be  of  tremendous  value  in  training  sonarmen  while  standing 
their  sonar  watch.   Another  question  posed  was  "What  kind 
of  operational  training  was  going  to  be  conducted?" 

At  this  point  it  is  necessary  to  distinguish  between 
operator  training  and  operational  training.   Operator 
training  is  defined  as  familiarization  training  centered 
around  sonar  equipment  functions  and  modes,  operation  of 
controls  and  switch/dial  settings  and  preliminary  operational 
adjustments.   On  the  other  hand  operational  training  is 
training  in  the  effective  utilization  of  the  available 
system  capabilities  to  accomplish  specific  tasks  such  as 
search,  track,  and  classification  procedures,  detection 
recognition  and  general  tactical  procedures.   Later, 
Commander,  Submarine  Development  Group  Two  asked  the 
question  "How  can  you  prove  that  training  had  occurred?" 
This  led  Mr.  Brown  to  investigating  operational  team  training 
concepts.   A  literature  search  and  discussions  with  sonar 
fleet  personnel  and  instructors  eventually  led  to  a  contract 
for  OBT  training  materials.   Several  technical  improvements 
were  then  made  to  the  OBT  including  switching  from  analog 
to  digital  displays. 

By  1975  the  improvements  to  the  on-board  trainer  and 
the  training  materials  had  been  completed.   At  this  time 
a  sea  trials  Operational  Evaluation  (OPEVAL)  on  the  U.S.S. 
WILLIAM  H.  BATES  (SSN-680)  was  conducted  with  structured 
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training  and  performance  evaluation.   Through  this  OPEVAL 
it  was  proved  that  training  did  occur.   Therefore,  the 
question  posed  earlier  by  the  Commodore  had  been  answered. 
Although  the  sea  trials  had  been  evaluated  an  overall  success, 
the  OBT  was  judged  as  not  very  maintainable  or  reliable.   A 
NAVSEA  decision  was  made  to  not  go  with  any  production  buy. 

Several  studies  were  conducted  showing  that  the  on-board 
trainer  could  interface  with  Digital  Multi-Beam  Steering 
(DIMUS)  sonar  systems.   The  AN/BQQ-5  had  a  training  mode 
but  the  program  office  would  not  buy  into  the  on-board 
trainer  even  though  it  had  been  shown  that  the  OBT  could 
interface  with  a  DIMUS  system.   Therefore  the  SSN  community 
never  received  the  on-board  trainer. 

In  1976  Mr.  Brown  approached  Strategic  Systems  Project 
Office  (SSPO)  with  the  operational  team  training  concept 
since  it  was  developing  a  land  based  Sonar  Operational  Trainer 
(SOT).   SP-15  gave  Mr.  Brown  $25,000  to  do  a  pilot  program 
for  operational  material  for  the  SOT.   By  this  action  Mr. 
Brown  had  sold  the  concept  of  operational  training  materials. 
SP-15  also  looked  at  the  on-board  trainer  for  SSBN's  and 
decided  to  acquire  them.   Now,  SOT  training  materials  and 
OBT  training  materials  were  to  be  developed  and  integrated 
for  ship  and  shore-based  training  by  NUSC.   It  was  at  this 
time,  October  19  76,  by  a  Memorandum  of  Agreement,  Strategic 
Systems  Project  Office  (SSPO)  assigned  to  the  Naval  Under- 
water Systems  Center  (NUSC)  the  functions  and  responsibilities 
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of  Principal  Developing  Activity  (PDA)  for  the  Sonar  Opera- 
tional Training  and  Assessment  Program  (SOTAP)  [Ref.  2]. 
Contracting  Officer  responsibilities  for  the  program  procure- 
ments to  support  SOTAP  was  delegated  by  NUSC  to  Naval 
Regional  Procurement  Office  (NRPO)  Philadelphia,  Newport 
Division. 

As  the  primary  Requiring  Activity,  SSPO  would  provide 
program  policy  direction  and  funding,  establish  and  main- 
tain applicable  training  specifications,  and  monitor 
overall  program  effectiveness.   As  PDA,  NUSC  would  develop, 
introduce,  and  maintain  all  program  materials.   These 
materials  would  implement  the  integration  of  Sonar  Opera- 
tional Trainers  (SOT) ,  On-Board  Submarine  Ocean  Acoustic 
Trainers  (SOAT) ,  and  AN/BQR-21  Unit  Lab  Trainer  (ULT)  into 
a  system  operational  training  on  SSBN  sonars,  and  operational 
assessment  of  both  sonar  and  combined  sonar/fire  control 
teams . 

Management  of  the  SOTAP  at  NUSC  would  be  the  responsi- 
bility of  the  Submarine  Sonar  Product  Line  (Code  32) .   To 
ensure  proper  integration  between  training  device  and  train- 
ing material  developments,  a  special  Program  Office  (Code 
3293)  was  established  within  the  Product  Line  to  manage  all 
SSBN  sonar  operational  training  related  programs.   In  view 
of  the  extensive  need  for  fleet  interaction,  a  Program 
Officer  billet  was  obligated  in  support  of  the  Program 
Office.   In  accordance  with  the  SOTAP  Memorandum-of -Agreement , 
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NUSC  will  contract  out  a  substantial  portion  of  the 
Program's  material  development  and  maintenance  efforts. 
The  program  participants  and  their  relationship  to 
the  program  are  listed  below: 

1.  COMSUBLANT/COMSUBPAC   -   Operational  Requirements 

2.  SSPO  -   Program  Sponsor 

3.  NUSC  -   Principal  Development  Activity 

(SOT,  SOTAP) 

4.  NAVSEA  -   Principal  Development  Activity 

(SOAT) 

5.  TRAFAC  -   SSBN  Shipboard  and  Off -Crew 

Training  Facilities 

In  1977  with  a  budget  of  $100,000,  NUSC  was  tasked  to 
develop  a  new  set  of  OBT  training  materials  for  the  SSBN's. 
In  their  final  form  these  materials  were  called  Exercise 
Controller  Guides  (ECG) .   In  August  of  1977  the  OBT  was 
installed  aboard  the  U.S.S.  SIMON  BOLIVAR  (SSBN-641)  and 
during  sea  trials  a  Technical  Evaluation  (TECHEVAL)  on  the 
hardware  was  successful.   In  September-October  during  patrol 
an  OPEVAL  with  an  Operational  Test  and  Evaluation  Force 
(OPTEVFOR)  rider  was  conducted  with  the  ECG.   The  OPEVAL 
was  successful.   To  quote  the  Commanding  Officer,  Cdr .  M.  J. 
DeHaemer,  "The  ECG  is  an  outstanding  document  in  support  of 
the  OBT.   The  format  and  underlying  concepts  are  sound  and 
it  was  demonstrated  to  me  during  OPEVAL  that  the  training 
method  if  very  effective  ..." 

The  foregoing  illustrates  the  current  state-of-the-art 
in  submarine  sonar  operational  programs  and  indicates  that 
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further  development  efforts  are  necessary  to  complete  the 
specific  needs  of  the  Sonar  Training  and  Assessment  Program 
(SOTAP) . 

B.   PURPOSE  OF  THE  STUDY 

The  objectives  of  this  study  are: 

1.  To  select  an  Information  Flow  and  Analysis  System 
(IFAS)  for  the  Sonar  Operational  Training  and  Assessment 
Program  (SOTAP) . 

2.  To  delineate  the  present  real  need  for  sonar  opera- 
tional training  programs. 

3.  To  describe  some  of  the  consequences  of  applying 
a  management  organization  and  principles  geared  to  the 
development  of  weapons  systems  to  the  development  of 
information  systems. 

4.  To  propose  a  systems  approach  for  the  acquisition 
of  a  basic  management  information  system. 

5.  To  identify  and  choose  an  information  system 
alternative  available  to  the  SOTAP  management,  and  propose 
recommendations  that  will  be  useful  in  implementing  the 
SOTAP  program  IFAS. 

This  study  focuses  on  broad  management  and  organizational 
relationships,  and  therefore  deliberately  avoids  to  the 
maximum  extent  possible,  the  more  technical  aspects  of 
computers  and  computer  utilization. 
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C.   METHOD  OF  RESEARCH 

The  basic  procedural  method  utilized  to  accomplish 
the  objectives  in  this  investigation  consisted  of  the 
following: 

1.  A  literature  review  in  the  areas  of  management 
information  systems,  training  information  management, 
training  data  base,  data  base  management,  training  data 
management,  technology  transfer,  and  government  directives 
was  made  in  order  to  provide  a  broad  background  in  manage- 
ment practices  of  information  systems  development. 

2.  Three  trips  and  numerous  phone  calls  were  used  in 
conducting  personnel  interviews  of  program  participants  and 
other  personnel  to  expand  upon  the  meager  amount  of  data 
available  concerning  team  training  concepts,  and  to  obtain 
their  expert  opinion  on  SSBN  submarine  sonar  operational 
training.   The  interviews  were  conducted  informally  with  no 
set  pattern  being  followed.   They  were  tailored  to  the 
interviewee  and  were  intended  to  provide  the  researcher 
with  an  insight  into  the  atmosphere,  attitude  and  functions 
of  the  various  activities  being  interviewed  and  to  provide 
pertinent  information  concerning  the  sonar  personnel.   The 
goal  was  to  establish  a  rapport  with  the  interviewee  and 

to  obtain  candid  information. 

3.  The  information  was  compiled,  then  analyzed. 
Chapter  II  delineates  the  need  for  sonar  operational 

training  programs,  showing  how  advances  in  technology, 
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personnel  shortages  and  non-continuous  operational  periods 
at  sea  for  Fleet  Ballistic  Missile  (FBM)  Sonar  Technicians 
have  led  to  the  institution  of  the  SOTAP  program.   Chapter 
III  discusses  weapons  systems  and  information  systems  and 
the  problems  that  could  occur  if  management  does  not  realize 
the  basic  differences.   Chapter  IV  proposes  a  systems 
approach  for  the  acquisition  of  a  basic  management  infor- 
mation system.   Chapter  V  presents  the  alternatives  that 
are  available  to  acquire  a  management  information  system 
from  the  author's  viewpoint.   Special  attention  is  devoted 
to  the  present  information  system,  Personnel  and  Training 
Evaluation  Program  (PTEP) ,  which  is  already  established  for 
certain  rating  groups  onboard  the  Fleet  Ballistic  Missile 
(FBM)  Submarines.   PTEP ' s  information  handling  system  con- 
version from  a  "batch  process"  to  an  on-line  real-time 
capability  under  the  Versatile  Training  System  (VTS)  by 
Fiscal  Year  1978  is  presented.   Chapter  VI  gives  conclusions 
and  recommendations  derived  from  the  study. 

Appendix  A  shows  a  systems  overview  drawing  illustrating 
the  information  systems  development  process  proposal. 
Appendix  B  paraphrases  the  important  portions  of  Digital 
Equipment  Corporation's  sales  brochure  on  its  Resource 
Sharing  Timesharing  System/Extended  (RSTS/E) ,  the  data 
management  system  used  by  VTS.   Appendix  C  is  the  currently 
used  PTEP  optically  scanned  data  scoring  form. 
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II.   NEED  FOR  SONAR  OPERATIONAL  TRAINING  PROGRAMS 

The  goal  of  this  chapter  is  to  show  that  a  real  need 
exists  for  sonar  operational  training  programs  even  though 
the  existence  of  the  SOTAP  program,  as  described  earlier 
in  the  background,  was  an  evolution  of  events  driven  by 
technology  (hardware)  rather  than  need. 

A.   TECHNOLOGICAL  ADVANCES 

Technological  change  has  gone  on  at  an  ever  accelerating 
pace,  especially  since  World  War  II.   Moreover,  technology 
has  changed  in  ways  that  differ  from  the  mechanistic,  mass- 
production  technology  that  until  quite  recently  was  considered 
to  be  all  there  was.   Not  only  has  the  time  required  to 
translate  a  basic  technical  discovery  to  commercial  produc- 
tion or  process  or  usage  decreased  to  a  few  years ,  but  also 
the  number  of  new  products  or  processes  is  increasing  exponen- 
tially [Ref.  3].   This  is  especially  true  in  the  Navy's  sub- 
marine sonar  area  as  reported  from  the  SOTAP  program  office 
where  the  complexity  of  the  Sonar's  has  increased  so  fast 
that  there  is  now  the  problem  of  how  to  operate  the  highly 
sophisticated  new  equipment  presently  on-board  the  submarines. 

The  Navy  has  tried  to  rectify  this  problem  by  using 
several  approaches.   One  requires  the  sonarmen  to  attend 
courses  taught  by  the  contractor  on  the  new  sonar  equipment. 
For  the  most  part  though,  these  factory  schools  have  taught 
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the  sonarmen  the  big  systems  viewpoint  or  what  the  sonar 
equipment  "can  do"  and  not  "how  to  operate"  the  sonar  to 
accomplish  different  functions  such  as  searches,  detection 
recognition,  tracking,  classifications,  etc.   Another 
approach  used  with  the  fast  attack  submarines  emerging  from 
the  shipyards  is  to  send  a  team  of  highly  qualified  per- 
sonnel to  the  submarine  to  conduct  a  six-day  intensified 
training  program  on  the  new  sonar  suite  for  the  sonar 
technicians.   Classes  are  conducted  each  of  the  six  days 
starting  at  approximately  0800  hours  and  running  until 
approximately  2300  hours.   This  approach  has  helped  somewhat 
although  it  has  been  very  hard  on  the  sonar  technicians  with 
the  standing  of  duty,  making  final  alignment  checks,  fixing 
problems  with  their  sonar  equipment,  and  clean-ups  in  the 
eight  hours  left  in  each  day. 

B.   PERSONNEL  SHORTAGES 

The  main  concern  in  the  past  was  in  the  areas  of  nuclear 
reactor  and  ballistic  missile  technology  on  submarines 
[Ref.  4].   Now  with  an  active  sonar  technology  growth  there 
is  an  increased  emphasis  at  all  levels  in  the  newer  highly 
sophisticated  sonar  equipments.   Many  of  the  more  senior 
sonarmen  are  not  adjusting  to  the  technological  change. 
Many  of  them  don't  understand  the  new  technology.   They 
feel  that  they  have  survived  in  the  past  with  the  older 
equipment  and  can  in  the  future. 

The  submarine  environment  itself  is  a  contributor  to 
personnel  shortages.   First  of  all,  not  everyone  can 
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physically  qualify  for  submarine  duty.   Although  physically 
qualified  for  the  Navy,  sailors  must  undergo  special  physi- 
cal examinations  for  submarine  duty.   Part  of  the  physical 
test  is  done  in  the  submarine  escape  training  tank.   Filled 
with  over  100  feet  of  water,  it  simulates  conditions  that 
would  exist  on  a  sunken  submarine.   Future  submariners  must 
successfully  ascend  from  50  feet  to  the  top  of  the  tank 
using  a  special  apparatus  (Steinke  hood)  for  breathing 
[Ref.  5].   The  sailor  must  also  pass  a  rigorous  submarine 
radiation  physical  administered  by  a  designated  submarine 
medical  officer. 

Second,  there  are  psychological  aspects  to  consider. 
A  phychological  factor  especially  evident  in  submarines  is 
claustrophobia.   In  an  SSBN  submarine  the  sailors  are  closed- 
in  and  submerged  for  the  entire  patrol  living  in  small, 
cramped  quarters. 

Separations  aren't  easy  and  are  especially  difficult  for 
the  wife,  parents,  or  friends  of  a  submarine  crew  member, 
not  only  because  of  the  frequency  and  length  of  the  separa- 
tions, but  also  because  of  necessary  restrictions  on  active 
communication  between  crew  member  and  friends.   Once  the 
boat  departs  for  patrol  a  crew  member  cannot  call,  write, 
transmit  messages,  or  send  a  telegram;  his  wife  or  friends 
can  send  him  only  a  few  20-word  "f amilygrams"  (five  during 
a  Fleet  Ballistic  Missile,  FBM,  underwater  patrol). 

There  is  good  reason  for  the  restrictions  on  communica- 
tions.  Successful  submarine  operations  depend  heavily 


19 


upon  secrecy.   The  SSBN  submarines  are,  in  effect,  mobile 
missile  bases.   Their  sixty  to  seventy  day  maneuvers  —  trial 
runs  for  a  situation  everyone  hopes  will  never  occur  —  must 
be  clandestine;  the  boats  do  not  surface,  they  do  not  pull 
into  port. 

In  the  SSBN  submarine  community  the  commitments  (i.e. 
an  at-sea  deterrent  force  with  weapons  covering  targets) 
mean  extended  work  days,  and  more  "midnight  oil"  in-port 
to  insure  the  at-sea  readiness  states  that  are  necessary. 
Most  people  understand  the  necessity  for  increased  working 
hours  and  unexpected  deployments  when  associated  with  a 
real  crisis.   But,  for  many,  the  call  for  sacrifice  has 
become  routine  and  long-term,  and  the  reasons  are  not  always 
apparent.   To  work  the  civilian  overtime,  the  price  is 
paid  in  increased  wages  (double-time,  time-and-a-half, 
etc.),  but  not  so  with  the  sailor.   Based  upon  the  author's 
experience  and  interviews  with  submarine  personnel,  it  is 
the  author's  opinion  that  the  price  is  paid  in  the  long 
run.   One  price  is  the  lack  of  adequate  retention.   Further- 
more, correction  of  our  retention  problem  is  aggravated  by 
the  problem  itself.   Shortages  mean  more  work  and  worse 
roatation  schedules,  making  for  further  and  worse  shortages. 
On  top  of  this  the  sonar  technicians  in  the  last  few  years 
have  seen  their  proficiency  pay  go  to  nothing  along  with 
other  actual  and  threatened  military  benefit  reductions. 
A  listing  of  military  benefit  reductions  since  Fiscal  Year 
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197  3  can  be  found  in  Ref.  6.   Many  SSBN  submarines  currently 
have  to  resort  to  non-sonar  technician  watchstanders  in 
sonar  to  meet  operational  requirements. 

C-   OTHER  PROBLEMS 

SSBN  submarines  are  designed  for  90-day  patrols,  all 
under  water;  therefore  each  ship  is  manned  by  two  complete 
crews,  designated  as  the  blue  crew  and  the  gold  crew.   When 
a  ship  returns  from  a  patrol  manned  by  the  blue  crew,  the 
gold  crew  is  ready  to  take  the  ship  to  sea  again.   This 
presents  the  problem  of  non-continuous  operational  periods 
of  time  at  sea  for  each  crew  that  is  peculiar  only  to  SSBN's. 
This  results  in  the  opinion  of  the  author  in  an  operational 
loss  of  learning  which  particularly  affects  the  more  junior, 
unexperienced  part  of  the  crew.   To  reduce  this  loss  of 
learning,  the  SSBN,  before  going  to  sea  on  sea  trials,  con- 
ducts a  "fast  cruise."   This  is  a  period  of  several  days 
moored  alongside  the  tender.   During  this  time  the  submarine 
simulates  conditions  at  sea  and  conducts  the  type  of  opera- 
tions that  would  be  conducted  at  sea  for  two  reasons.   One 
reason  is  to  ensure  all  the  equipment  aboard  is  working 
properly  while  the  other  reason  is  to  re-train  the  crew  in 
the  various  submarine  operations. 

The  mission  of  the  SSBN  on  patrol  is  to  act  as  a  strate- 
gic deterrent  against  our  enemies.   The  SSBN  is  to  submerge, 
remain  undetected,  and  ready  at  all  times  to  fire  all  their 
missiles  within  minutes  if  ordered  to  do  so.   Once  a  contact 
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is  detected,  if  possible,  the  SSBN  will  use  all  measures 
available  to  avoid  the  potential  threat.   Therefore,  the 
mission  and  types  of  operations  of  an  SSBN  are  not  conducive 
to  staying  experienced  in  all  sonar  operational  characteristics. 

Other  SSBN  sonar  team  performance  current  training 
problems  obtained  through  the  SOTAP  Program  Office, 
Principal  Investigator,  are  listed  below: 

1.  Formal  training  focused  on  "How  equipment  operates" 
rather  than  "How  to  operate  the  equipment" 

2.  Non-standardized  team  training  at  the  off -crew 
training  sites 

3.  No  reliable  team  performance  evaluation 
capabilities 

4.  Current  team  training  devices  are  obsolete 

5.  No  operational  training  information  flow  between 
training  sites. 
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III.   COMPARISON  OF  WEAPONS  SYSTEMS 
AND  INFORMATION  SYSTEMS 

Although  the  management  organization  for  the  development 
of  information  systems  in  industry  and  government  is  very 
different  from  that  in  the  military,  traditional  experience 
with  the  acquisition  of  hardware  systems  influences  and  pre- 
vades  both  areas.   To  bring  out  as  forcefully  as  possible 
how  this  influence  occurs  and  the  management  problems  derived 
thereby  for  the  development  of  information  systems,  the  rest 
of  this  chapter  is  based  on  a  comparison  of  the  basic  charac- 
teristics of  weapons  systems  with  those  of  information 
systems.   This,  of  course,  represents  the  extreme  case  since 
the  development  of  weapons  systems  by  the  military  occurs 
under  conditions  of  unusual  uncertainty,  by  contrast  with 
nonmilitary  hardware  systems,  and  in  the  context  of  a  highly 
formalized  managerial  structure  and  process. 

A  listing  of  the  basic  differences  between  weapons  systems 
and  information  systems  is  listed  in  Table  III-l  [Ref.  7]. 
It  should  be  borne  in  mind  that  this  list  is  highly  simplified 
for  the  sake  of  the  following  explication.   The  author  can 
deal  here  only  with  the  more  obvious  differences.   There  are 
many  additional  differences  in  such  areas  as  system  testing, 
quality  control,  and  maintenance,  the  cumulative  effect  of 
which  has  important  implications  for  the  management  of  the 
system  development  effort.   These  additional  differences  will 
not  be  addressed. 
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TABLE  III-l 


BASIC  DIFFERENCES  BETWEEN 


WEAPONS  SYSTEMS  AND  INFORMATION  SYSTEMS 


Weapons  Systems  Information  Systems 

1.  Multiple  users  1.  Single  users 

2.  Many-of-a-kind  2.  One-of-a-kind 

3.  Model  changes  3.  Planned  evolutionary 

change 

4.  Hardware  state-of-the-art  4.  Software  state-of-the- 
is  critical  art  is  critical 

5.  High  cost/effectiveness  5.  Low  cost/effectiveness 
ratio  ratio 

6.  Operational  independence  6.  Functional  integration 
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Bearing  in  mind  the  basic  differences  between  weapons 
systems  and  information  systems  as  shown  in  Table  III-l  the 
rest  of  the  chapter  considers  the  consequences  of  applying 
a  management  organization  and  principles  geared  to  the  develop- 
ment of  weapons  systems  to  the  development  of  information 
systems.   The  identifying  numbers  of  the  following  sections 
correspond  to  the  numbers  in  the  table. 

1 .   The  Information  System  is  Custom-Made  to 
Fit  the  User 

The  same  weapon  or  hardware  system  can  be  used  equally 
effectively  by  a  variety  of  users.   A  strategic  missile  can  be 
employed  by  different  services  within  the  same  country  or  by 
different  countries.   The  same  is  true  of  ships.   Such  is  not 
the  case  with  information  systems.   An  information  system  is 
tailor-made  to  fit  the  needs,  objectives,  and  requirements  of 
a  unique  user.   Each  military  command  and  each  industrial 
enterprise  needs  information  of  a  special  kind.   In  the 
industrial  computer  applications  such  as  payroll  accounting, 
inventory  control,  production  control,  banking,  insurance, 
transportation,  etc.,  an  examination  of  the  details  of  these 
applications  in  similar  areas  would  still  show  basic  differ- 
ences such  as  differences  in  computer  programs,  in  the  format 
and  content  of  displays  and  reports,  in  the  construction  of 
the  data  base,  in  the  relationships  among  system  components, 
and  in  the  use  of  human  beings  as  elements  of  the  system. 

Since  each  information  system  is  custom-made  to  meet 
the  special  needs  of  a  single  user,  the  developer  must  study 
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the  operations  of  the  current  system,  assuming  there  is  one, 
in  order  to  clarify  the  user's  problems,  to  determine  his 
needs  and  objectives,  and  to  establish  preliminary  system 
requirements.   The  difficulty  in  study  is  obtaining  complete 
and  accurate  information  on  all  relevant  areas  of  systems 
operations.   Equal  in  importance  to  the  study  of  the  user's 
current  system  is  the  study  and  analysis  of  the  system's 
future  requirements. 

2.   Many  of  a  Kind/One  of  a  Kind 

Many  basic  differences  in  weapons  and  software  systems 
which  have  a  profound  impact  on  management  stem  from  the  fact 
that  weapons  systems,  with  some  notable  exceptions,  are  usually 
produced  in  large  numbers  from  a  prototype  model.   Information 
systems  are  one-of-a-kind,  that  is,  only  one  operational 
system  is  ever  developed  from  the  design.   The  information 
system  is  not  a  mass  produced  article.   But  the  fundamental 
difference  pointed  out  here  between  weapons  systems  and  infor- 
mation systems  remains  —  current  management  organization  and 
concepts  are  geared  for  the  most  part  to  a  tradition  of  mass 
production,  not  the  production  of  one-of-a-kind  items. 

A  different  attitude  toward  system  testing  is  demanded 
of  the  manager  because  of  the  inherent  differences  between 
hardware  systems  and  information  systems.   It  is  true  that 
weapons  systems  can  be  reduced  to  obsolescence  by  technolo- 
gical advances.   But  as  rapid  as  technological  change  is,  no 
one  will  claim  that  it  occurs  on  a  daily  basis.   In  any  case, 
the  physical  environment  for  which  the  weapons  system  was 
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designed  does  not  change.   Thus,  it  is  possible  to  subject 
the  weapon  system  to  rigorous  tests  under  controlled  condi- 
tions to  determine  its  reliability  and  design  validity. 
Such  is  not  the  case  for  information  systems.   The  information 
system  must  be  tested  for  the  full  range  of  operational  possi- 
bilities in  an  environment  which  may  be  undergoing  change  on 
a  daily  basis.   The  ability  of  the  information  system  to 
adapt  to  such  changes  is,  in  itself,  a  test  variable.   To 
provide  adequately  for  such  system  testing  requires,  first, 
understanding  the  need  and,  second,  alloting  the  necessary 
resources  to  do  the  job. 

The  one-of-a-kind  information  system  poses  many  special 
problems  for  training  which  do  not  exist  for  many-of-a-kind 
systems.   Training  must  be  conducted  for  the  one-of-a-kind 
system  without  interfering  with  on-going  operations.   It  might 
be  necessary  to  design  a  simulation  capability  into  the  opera- 
tional system  in  such  a  way  that  both  operations  and  training 
can  be  conducted  simultaneously. 

Finally,  it  must  be  mentioned  with  respect  to  the  many- 
of-a-kind/one-of-a-kind  differentiation  the  managerial  head- 
ache, shared  with  the  developer,  of  phasing  in  the  new  system 
to  assume  operational  responsibility  without  interfering  with 
on-going  activities  [Refs.  8  and  9].   Few  operations,  military 
or  nonmilitary,  can  afford  to  close  up  shop  for  a  period  of 
time,  however  short,  in  order  to  make  the  shift  from  one 
system  to  another.   Must  the  user  suffer  through  a  period  of 
degraded  operational  capability  while  the  new  system  is  being 
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phased  in  and  the  old  one  phased  out?   In  the  one-of-a-kind 
system  this  is  a  major  managerial  dilemma.   Thus,  the  phase- 
over  period  is  a  critical  one,  involving  both  training  and 
operations,  which  call  for  much  research,  exploratory  effort, 
planning,  and  design  in  order  to  ensure  a  smooth  transition. 
3 .   Model  Changes/Planned  Evolution 

Another  basic  difference  between  hardware  systems 
and  information  systems  is  to  be  found  in  the  nature  of  their 
change  and  replacement  through  time.   Weapons  systems  proceed 
through  what  is  called  "model"  changes,  whereas  in  information 
systems  changes  are  referred  to  as  "planned  evolution."   In 
the  case  of  weapons  systems,  the  initial  weapon,  if  it  changes 
at  all,  undergoes  a  series  of  incremental  modifications  as 
technology  improves  or  requirements  change,  but  the  final 
model  could  not  be  technically  implemented  when  the  program 
for  the  weapon  began.   Each  model  is  a  part  or  complete 
replacement  of  the  previous  one  although  earlier  versions  may 
continue  to  be  utilized  in  the  weapons  inventory.   A  typical 
example  of  model  changes  is  the  series  of  B-52  bombers. 
Similarly  for  missiles,  torpedoes,  etc.  each  subsequent  model 
incorporates  improved   capabilities  of  various  kinds  —  range, 
speed,  altitude,  reliability,  or  load  capacity. 

By  contrast  with  weapons  systems,  information   systems 
are  evolutionary  in  that  they  are  designed  and  implemented  in 
several  iterations  to  perform  information-processing  functions 
for  a  continuing  enterprise.   The  information  system  evolves 
through  a  planned  series  of  stages  or  phases  each  of  which 
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includes  the  addition  of  new  tasks   and  functions  which  may 
have  been  conceived  and  regarded  as  feasible  from  the  inception 
of  the  plan.   It  is  also  possible  that  functions  not  conceived 
during  the  original  planning  may  be  added  at  a  later  date,  but 
these  should  be  integrated  with  the  long-range  plan.   The 
system  as  it  exists  at  any  stage  or  phase  incorporates  earlier 
phases;  it  does  not  replace  them,  as  is  the  case  with  weapons 
systems,  although  the  same  functions  may  be  performed  by  more 
efficient  computer  programs  or  better  allocations  of  tasks 
among  men  and   machines. 

The  term  "evolution"  is  appropriate  for  information 
systems  also  in  that  they  are  adaptive  to  their  environment. 
An  information  system  has  the  capacity  to  adapt  itself  to 
changing  situations  and  the  capacity  to  learn  from  experience. 
These  capacities  are  provided  by  its  human  components,  who 
are  themselves  adaptable  and  capable  of  learning.   Modifica- 
tions to  the  system  are  made  through  an  on-going  dialogue 
between  system  users  and  designers.   As  they  apply  the  system 
and  gain  experience  with  it,  the  users  recommend  to  the 
designers  improvements  to  procedures,  computer  programs, 
displays,  etc.   Eventually,  by  means  of  "heuristic"  program- 
ming, information  systems  may  have  a  capacity  through  their 
computer  programs,  as  distinct  from  their  human  operators, 
to  improve  their  performance  by  an  inherent  adaptive  or 
learning  capability  [Ref.  10].   A  weapons  system  is  not 
adaptive  in  this  sense. 
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A  given  model  of  an  aircraft  or  a  missile  pushes 
the  hardware  state  of  the  art  to  the  limit.   A  given  stage 
or  phase  of  an  information  system  does  not  necessarily  reflect 
a  limit  of  the  computer  state  of  the  art.   It  may  reflect  a 
variety  of  other  factors,  such  as  the  desire  to  initiate  at 
least  a  modest  capability  as  soon  as  possible,  limited 
funding,  or  the  fact  that  the  user's  requirements  are  not 
clearly  known  so  that  the  ultimate  system  cannot  be  specified 
in  detail  immediately.   Also,  in  the  case  of  military  informa- 
tion systems,  the  rate  of  technological  change  and  of  changes 
in  mission  requirements  suggest  that  freezing  the  design  as 
final  at  any  given  stage  is  undesirable.   Hence,  a  modest 
beginning  is  made  by  using  an  initial  operational  capability 
with  the  understanding  that  later  phases  of  the  system  will 
incorporate  technological  changes  and  new  mission  requirements. 
But  the  final  operational  capability  for  the  information  system 
is  equivalent  to  that  of  the  entire  increment  of  models  for 
a  given  weapons  series. 

The  evolution  of  information  systems  raises  a  number 
of  other  questions  related  to  recent  changes  in  approach  to 
systems  acquisition  by  the  Department  of  Defense.   The  intimate 
relationship  which  is  necessary  between  the  user  and  the  soft- 
ware developer  during  the  requirements  and  design  phases  in 
the  development  of  information  systems  raises  doubts  about  the 
desirability  of  competitive  bidding  between  different  software 
developers.   A  frequent  complaint  of  users  is  that,  even  when 
one  developer  is  involved,  they  are  asked  the  same  question 
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about  their  operations  by  different  personnel  from  the  same 
development  organization.   Obtaining  information  about  the 
user's  daily  operations  as  a  basis  for  designing  the  new 
system  is  a  delicate  task  even  under  ideal  conditions.   It  is 
difficult  to  imagine  the  chaos  if  two  or  more  software 
competitors  were  simultaneously  engaged  in  obtaining  opera- 
tional information  and  conducting  operations  analyses. 
4 .   Hardware/Software  Sciences 

Studies  made  within  the  defense  establishment  of 
military  information  systems  and  the  private  sector  agree 
that  computer  technology  exceeds  at  the  present  time  our 
ability  to  put  together  the  most  effective  systems  [Ref .  8] . 
Hardware  systems  not  specifically  designed  for  military  use, 
such  as  satellites  and  research  rockets,  all  push  the  hard- 
ware state  of  the  art  in  such  areas  as  propulsion, guidance , 
miniaturization,  and  communications.   Although  information 
systems  could  profit  from  improvements  in  such  areas  as  core 
storage  capacities,  speed  of  operations,  display  devices,  and 
input/output  devices,  the  technological  limitations  in  these 
fields  do  not,  of  themselves,  constitute  insuperable  constraints 
on  the  design  of  contemporary  information  systems. 

The  incorporation  of  the  computer  as  the  basic  compo- 
nent in  large-scale  information  systems  to  assist  in  decision 
making  involves  the  designer  of  such  systems  in  a  host  of 
so-called  "soft"  sciences  such  as  human  relations,  management 
science,  psychology,  social  psychology,  sociology,  applied 
anthropology,  and  human  engineering.   All  these  sciences  are 
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necessary  in  the  design  of  information  systems  since  they 
contribute  to  the  understanding  of  the  behavior  of  human 
beings  as  individuals  and  as  members  of  groups.   Valid  per- 
formance measures  for  information  systems  in  which  human 
beings  and  group  dynamics  play  vital  roles  cannot  be  estab- 
lished if  the  human  and  group  factors  are  ignored.   By 
contrast,  in  the  design  of  weapons  and  other  types  of  hardware 
systems,  human  beings  and  groups  play  minor  or  nonexistent 
roles  [Ref.  11].   In  such  systems,  therefore,  the  relevant 
sciences  are  the  more  traditional  and  more  advanced  "hard" 
sciences  such  as  physics  and  chemistry. 

One  problem  area  is  the  types  of  skills  required  to 
produce  software  items.   The  typical  potential  user  of  an 
information  system  has  been  accustomed  to  buying  hardware. 
As  a  result,  he  is  familiar  with  the  types  of  specialists 
normally  involved  in  the  design  and  production  of  hardware 
elements.   He  knows  about  system  engineers,  system  analysts, 
and  operations  research,  or  at  least  he  has  heard  that  such 
specialists  and  fields  of  knowledge  make  contributions  to  the 
development  of  hardware  systems,  and  he  is  willing  to  pay  for 
these  skills.   But  it  is  not  uncommon  to  find  not  only  that 
the  typical  user  of  an  information  system  does  not  know  what 
kinds  of  sciences  play  a  role  in  the  design  and  production  of 
software,  but  also  that  he  may  have  a  bias  or  distinct  preju- 
dice against  "soft"  sciences.   Since  the  output  of  the  soft 
or  social  sciences  is  less  tangible  than  the  hard  sciences, 
the  user  tends  to  be  reluctant  to  pay  for  it. 
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The  role  of  experts  from  the  field  of  group  dynamics, 
a  branch  of  social  psychology,  may  serve  to  illustrate  the 
participation  of  nonhardware  scientist  in  a  particular  infor- 
mation system  development.   RAND  Corporation  investigated  the 
inadequate  performance  of  systems  with  human  beings  as 
components  and  developed  the  System  Training  Program  (STP) 
[Ref.  12].   One  of  the  so-called  STP  principles  emphasized 
by  RAND  researchers  was  the  provision  of  knowledge  of  results 
to  personnel  participating  in  the  training  exercises.   This 
knowledge  of  results  was  presented  in  a  "debriefing"  imme- 
diately following  the  exercise.   It  was  not  merely  enough  to 
solve  the  technical  problems  of  recording  trainee  performance, 
analyzing  the  results,  and  summarizing  them  in  some  meaningful 
fashion.   There  were  two  other  very  important  issues  which 
the  software  developer  had  to  resolve:   (1)  how  could  the 
results  of  the  exercises  be  presented  to  the  trainees,  and 
(2)  how  should  a  debriefing  be  conducted  to  ensure  maximum 
participation  by  all  trainees? 

These  issues  were  investigated  by  the  software  devel- 
oper's staff  of  experts  on  group  dynamics,  working  closely 
with  psychologists  familiar  with  learning  theory.   Experience 
with  the  training  program  had  shown  that  maximum  problem- 
solving  activity  on  the  part  of  the  trainees  did  not  occur  if 
the  exercise  results  were  presented  in  a  manner  which  the 
trainees  might  interpret  as  blame  fixing.   Also,  since  many 
of  the  operations  in  the  transmission  of  data  and  information 
during  the  exercises  were  invisible  to  both  the  observers  and 
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to  the  trainees,  it  was  evident  that  full  understanding  of 
what  had  occurred  during  the  exercise  depended  upon  creating 
an  atmosphere  in  the  debriefing  which  would  encourage  personnel 
to  talk  freely  about  the  actions  and  decisions  they  had  taken. 

How  do  you  persuade  people  to  talk  freely  about  their 
mistakes  in  front  of  their  peers  and  superiors?   How  do  you 
suggest  to  military  officers  that  maximum  participation  in  a 
debriefing  by  all  personnel  can  be  achieved  in  a  permissive, 
non- threatening,  non-blame-fixing  group  atmosphere?   How  do 
you  get  individuals  to  think  of  their  operational  environment 
with  a  system  perspective?   Research  on  these  issues  was 
conducted  by  the  group  specialists  and  psychologists  at  RAND 
and  manuals  on  the  proper  conduct  of  debriefings  were  pub- 
lished [Ref.  13];  and  training  programs  for  debriefing  offi- 
cers were  held  [Ref.  14]. 

Obviously,  research  activities  in  such  areas  as  group 
dynamics  and  the  relationships  between  displays  and  decision 
making  consume  scarce  resources  such  as  personnel,  funds,  and 
facilities.   It  takes  time  to  conduct  research,  to  publish 
the  results,  to  develop  the  specifications  for  displays,  and 
to  develop  orientation  and  training  programs  on  the  conduct 
of  debriefings.   The  professional  nonhardware  scientists 
participating  in  the  software  development  process  are  well 
aware  that  these  activities  are  necessary  to  maximize  system 
effectiveness,  but  it  is  up  to  the  management  of  the  users, 
procurement  agencies,  technical  agencies,  and  hardware 
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developers  to  understand  why  these  things  must  be  done 
to  provide  the  necessary  resources. 

Another  problem  area  is  the  lack  of  a  commonly 
accepted  set  of  terms  to  identify  software  items.   The 
distinctive  jargons  of  specialized  disciplines,  in  addition 
to  the  lack  of  consensus  on  the  identification  and  content 
of  software  products,  contribute  to  confusion  with  respect 
to  software  terminology  in  current  use.   Another  source  of 
confusion  is  the  fact  that  many  of  the  terms  used  to  refer 
to  software  products  are  borrowed  from  the  hardware  and 
weapons  development  fields. 

The  emergence  of  any  new  technology  is  always  accom- 
panied by  an  associated  jargon  specific  to  the  processes, 
activities,  and  objects  of  that  technology.   The  software 
field,  no  less  than  any  other,  has  its  own  needs  for  a  unique 
language.   The  fact  that  there  is  as  yet  no  common  agreement 
on  the  terminology  used  and  that  the  referents  of  the  terms 
change  through  time  reflect  the  early  stage  of  information 
system  technology.   Efforts  to  standardize  terminology  are 
being  pushed  within  the  data  processing  industry,  in  the  armed 
services,  and  also  within  the  Department  of  Defense. 
5 .   Cost/Effectiveness  Ratio 

As  the  cost  of  weapons  increases  exponentially  with 
their  growing  technological  complexity  and  sophistication, 
each  weapon  considered  for  the  national  inventory  must  be 
carefully  evaluated  on  the  basis  of  the  effectiveness  pur- 
chased for  each  dollar  invested.   Similarly,  an  information 
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system  must  be  evaluated  in  terms  of  the  effectiveness  bought 
for  a  military  command  by  the  investment  of  limited  funds. 
As  the  cost  of  both  hardware  systems  and  information  systems 
rises  steeply,  managerial  decisions  must  be  made  respecting 
the  allotment  of  limited  funds  for  more  and  better  weapons 
or  for  more  and  better  information  systems. 

When  examined  in  terms  of  absolute  dollar  value,  the 
price  of  an  information  system  may  appear  high,  paritcularly 
those  costs  accruing  during  the  preproduction  phases  of 
development.   There  are  two  points  to  be  considered  here. 
First,  the  funds  required  to  design  and  build  a  computer- 
based  information  system  are  amortized  over  the  years  in 
which  successor  systems  are  designed  and  built.   The  experi- 
ence, knowledge  and  software  products  gained  during  the  con- 
struction of  the  system  are  passed  on  to  subsequent  systems. 
Second,  an  information  system  provides  the  user  with  a  very 
large  amount  of  effectiveness  for  the  money  it  costs  when  this 
effectiveness  is  measured  over  the  life-span  of  the  system. 
With  appropriate  modifications,  given  the  planned  evolutionary 
approach,  the  system  will  last  for  the  life-span  of  the  user. 
Funds  alloted  for  the  design  and  production  of  weapons  systems, 
by  contrast,  are  lost  as  soon  as  those  weapons  systems  are 
fired,  as  in  the  case  of  missiles,  or  become  obsolete  in 
approximately  four  or  five  years  due  to  a  newer  technological 
threat.   It  is  meaningless,  therefore,  to  compare  weapons 
systems  with  information  systems  in  terms  of  absolute  dollars. 
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6 .   Independent  Operation/Operational  Integration 

The  typical  weapons  system  is  relatively  self-contained 
and  self-sufficient.   It  is  this  quality  of  independence  of 
the  system  from  the  user  which  makes  it  possible  for  the  same 
weapon  to  be  used  by  various  services  within  the  same  nation 
as  well  as  by  different  nations,  assuming  the  existence  of 
an  adequate  technological  base.   By  contrast,  the  information 
system  is  not  self-sufficient  or  self-contained.   This  char- 
acteristic interdependence  of  information  systems  is  referred 
to  in  the  technical  literature  as  "functional  integration" 
and  "technical  integration."   "Functional  integration"  refers 
to  the  operational  interdependence  of  associated  systems. 
"Technical  integration"  refers,  as  the  term  implies,  to  the 
compatible  linkages  of  data  and  equipment  in  the  mechanical 
or  electronic  sense. 

In  the  past,  the  influence  of  weapons  systems  and  a 
traditional  hardware  orientation  has  tended  to  emphasize 
technical  integration  at  the  expense  of  functional  integra- 
tion [Refs.  4  and  15].   There  are  other  reasons,  too,  why 
functional  integration  is  likely  to  be  relatively  neglected, 
such  as  the  sensitivities  of  existing  organizations  to  juris- 
dictional problems.   For  understandable  reasons  the  decentral- 
ized department  manager  resists  the  trend  toward  "recentral- 
ization"  made  possible  by  computer  based  management  systems. 
Early  in  the  1960 's  an  important  series  of  technical  studies 
of  the  problems  associated  with  the  development  of  information 
systems  stressed  the  point  that  the  key  problem  facing 
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management  in  the  defense  establishment  is  not  merely  tech- 
nical integration,  but  functional  integration  as  well  [Ref.  16] 

Functional  interdependence  of  information  systems 
affects  the  devoloper  in  other  ways.   In  the  course  of  system 
design,  for  example,  the  design  effort  is  necessarily  con- 
strained by  interface  considerations.   At  each  point  of  inter- 
face, ideal  design  decisions  may  have  to  give  way  to  compro- 
mises in  order  to  establish  the  necessary  linkage  with  other 
systems.   In  such  cases  the  developer  may  see  the  need  for 
the  coordination  of  design  decisions  with  other  agencies  and 
organizations  outside  the  immediate  jurisdiction  of  his  con- 
tract, but  neither  the  user  nor  these  agencies  and  organiza- 
tions may  recognize  the  need  or  be  willing  to  devote  the  time, 
and  effort  to  respond  to  it. 

In  summary  this  systematic  comparison  of  weapons 
system  characteristics  with  information  systems  characteris- 
tics brings  out  the  extent  to  which  contemporary  management 
of  users,  procurement  agencies,  and  technical  agencies  may 
be  utilizing  an  irrelevant  system  model  for  the  acquisition  of 
information  systems. 


IV.   SYSTEM  DEVELOPMENT  OF  AN  INFORMATION  SYSTEM 

In  the  course  of  its  development  every  large-scale  informa- 
tion system  must  pass  through  a  sequence  of  phases  in  its  life 
history.   The  use  of  the  term  "phase"  in  the  context  of 
systems  development  should  be  qualified.   Only  in  a  high  level 
of  abstraction  is  there  distinguishable  phases  of  development 
and  that  they  represent  a  logical  and  temporal  sequence.   In 
some  cases,  the  primary  process  within  a  phase  which  gives 
that  phase  its  name,  such  as  requirements  or  design,  is  also 
an  activity  or  function  which  is  performed  in  other  phases 
as  well.   The  system  requirements,  for  example,  must  be  deter- 
mined before  the  initial  design  activity,  but  the  determination 
of  requirements  does  not  terminate  at  any  specific  phase. 
Throughout  the  course  of  the  development  of  a  system,  old 
requirements  are  constantly  undergoing  refinement  while  more 
detailed  requirements  are  being  generated.   When  the  system 
first  becomes  operational,  actual  experience  with  it  may  give 
rise  to  new  requirements.   Changes  in  the  system's  environment 
or  in  technology  may  also  result  in  the  creation  of  new 
requirements.   Similarly,  system  design,  in  addition  to 
serving  as  a  name  for  a  logical  and  temporal  phase  which 
follows  the  requirements  phase,  is  also  a  function  which  is 
carried  out  repetitively  at  different  levels  of  the  system 
development  process. 

Four  project  phases  will  be  discussed  in  this  chapter. 
Many  authors  on  the  systems-development  process  have  also 
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outlined  the  phases  of  a  systems  project.   Laden  and  Gilder- 
sleeve  have  designated  the  first  of  these  as  a  Survey,  which 
is  followed  by  Systems  Investigation  (data  gathering) , 
Systems  Design,  Programming,  Filemaking,  Clerical  Procedures, 
Systems  Testing,  and  Parallel  Running  [Ref.  17].   Their  Survey 
and  Systems  Investigation  covers  what  the  author  chooses  to 
call  Requirements  (defining  the  need,  generating  a  proposal, 
feasibility  assessment,  project  start-up).   Systems  Design, 
Programming,  Filemaking,  Clerical  Procedures  corresponds  to 
Development  (Detailed  System  Design) ;  and  Systems  Testing, 
Parallel  Running  is  the  same  as  the  author's  Implementation. 
To  emphasize  the  total  life-cycle  concept  the  Utilization 
Phase  was  added. 

Although  Laden  and  Gildersleeve  primarily  addressed 
batch-processing  systems  in  their  book,  Head  outlines  the 
basic  development  process  steps  found  in  real-time  systems  as 
Preliminary  Technical  Planning,  Record  Specification,  Program 
Specification,  Programming,  System  Testing  and  Conversion  and 
Operation  [Ref.  18].   Seemingly  inevitable  parallels  to  all 
these  quite  similar  project  structures  can  be  found  on  further 
investigation  [Refs.  19,  20  and  21].   This  being  so,  perhaps 
the  author  can  safely  proceed  to  discuss  these  phases  as  they 
are  variously  described  in  greater  detail,  confident  that, 
though  the  names  are  different,  the  substance  is  essentially 
the  same.   Appendix  A  contains  a  systems  overview  drawing 
showing  the  information  systems  development  process. 
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It  is  not  the  intention  of  the  following  paragraphs  to 
present  a  detailed  checklist  of  the  contents  of  each  phase  of 
a  major  project.   This  has  been  done  for  many  different 
types  of  projects  more  than  adequately,  and  the  reader  is 
referred  to  several  sources  [Refs.  17,  18,  19,  20,  21,  and  22] . 
Rather  the  author  has  tried  to  survey  the  available  literature 
and  develop  a  basic  systems  approach  oriented  towards  the 
possible  acquisition  needs  of  a  Navy  project  for  the  acquisi- 
tion of  a  computer-assisted  management  information  system  (MIS) 

A.   PHASE  ONE  -  REQUIREMENTS 
1 .   Pre-Proposal 

The  translation  of  a  recognized  need  or  opportunity 
in  the  systems  area  into  preliminary  informal  "working  papers" 
as  a  basis  for  further  study  and  definition. 

Ideas  for  systems  work  may  originate  anywhere  in  the 
organization,  most  frequently  in  the  potential  using  organiza- 
tion itself.   Definition  of  needs  and  opportunities  is  not  at 
this  stage  expected  to  have  taken  into  account  related 
efforts,  feasibility,  or  availability  of  resources.   It  is 
necessary  first  to  define  the  problem  area  and  its  magnitude 
in  order  that  the  user  can  place  it  in  the  context  of  his 
overall  objectives  in  the  systems  area,  and  decide  on  the 
relative  emphasis  he  wants  to  give  the  proposal.   Specifically, 
the  objectives  of  this  activity  are  as  follows: 

a.  Definition  of  the  problem  area. 

b.  Ranking  of  importance  to  user. 
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c.  Determination  of  the  amount  to  be  budgeted  for  a 
systems  effort  in  this  problem  area  for  the  coming  planning 
period. 

d.  Providing  a  basis  for  communicating  about  the 
problem  with  concerned  management  and  staff  people  both  in 
the  user  organization  and  outside  it. 

Certain  procedural  steps  that  should  be  followed  are: 

a.  At  an  early  stage  the  person  or  group  in  the  user 
organization  responsible  for  overseeing  and  co-ordinating 
systems  development  performed  for  the  organization,  assumes 
responsibility  for  the  pre-proposal  activity  even  though  the 
ideas  may  have  originated  elsewhere. 

b.  The  user's  systems  manager  (if  one  exists), 
governed  by  the  policies  established  by  his  superiors  for 
the  conduct  of  his  activities,  prepares  for  his  management 
the  information  necessary  for  them  to  make  certain  decisions. 
This  information  includes  the  description  of  a  potential 
project,  a  general  statement  of  its  potential  benefits  and 
impact  on  the  organization,  its  relationship  to  the  user's 
ongoing  developments  or  existing  systems,  its  suggested 
priority,  and  the  recommended  amount  of  budget  data  that 
should  be  reserved  for  further  work  in  the  area  over  the 
ensuing  budgetary  period. 

c.  The  management  of  the  user  organization  must  make 
a  decision  to  authorize  a  proposal  aimed  toward  establishing 
a  project,  based  on  the  recommendations  made  to  it  by  the 
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systems  manager.   It  must  decide  when  and  by  whom  this 
proposal  effort  is  to  be  conducted. 

The  delay  depicted  in  the  drawing  ensues  between  these 
two  activities  depending  on  the  priority  assigned  by  using 
organization.   If  given  a  high  priority,  further  action  may 
take  place  without  delay. 

2.   Proposal  Preparation 

The  conversion  of  internal  "working  papers"  of  the 
user  organization  into  a  systems  proposal  as  a  basis  for 
communicating  with  the  systems  organization  (if  one  exists). 

The  document  will  be  referred  to  here  as  a  "systems 
development  proposal,"  that  is,  the  user  will  propose  that 
the  systems  organization  undertake  to  develop  the  system 
described  in'  the  document.   There  is  no  intent  to  make  this 
document  conform  to  a  standard  set  of  ground  rules  with 
respect  to  form  and  content,  but  certain  guidelines  are 
suggested  to  facilitate  subsequent  study  and  negotiation. 
This  is,  therefore,  not  a  formal  procedure,  since  the  systems 
organization  ought  always  to  be  ready  to  discuss  a  user's 
requirements  when  the  user  feels  the  time  is  ripe  for  external 
consideration.   There  may  be  no  clearcut  division  between 
Pre-proposal  and  Proposal  Preparation  in  Phase  One. 

The  systems  proposal  as  a  minimum  should  include  the 
following: 

a.   A  description  of  the  system  in  terms  of  management 
functions  included  or  signficantly  changed. 
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b.  A  brief;,  preliminary  description  of  the  proposed 
systems  concept,  on-line,  batch,  type  of  communications,  mode 
of  input/output,  etc. 

c.  A  qualitative  statement  of  the  benefits  expected, 
in  order  of  importance  (cost  avoidance,  improved  service, 
improved  timeliness,  increased  accuracy,  etc.). 

d.  Relationships  to  any  other  of  the  user's  systems 
in  operation  or  under  development,  and  to  any  other  systems 
(if  known) . 

e.  The  amount  currently  budgeted  for  the  proposed 
system. 

f.  A  statement  of  the  importance  of  the  need  relative 
to  other  existing  or  forthcoming  systems'  development  and  to 
other  management  plans  of  the  user. 

The  proposal  may  also  include  other  information  that 
would  utlimately  have  to  be  developed  for  final  management 
approval.   This  feasibility  information  should  be  quantitative 
and  specific,  and  should  deal  with  cost/benefit,  technical 
risk,  resource  requirements,  work  plan,  etc. 

The  procedural  steps  are  presented  below. 

a.   The  user's  management  must  decide: 

(1)  When  it  wants  to  present  the  systems  proposal 
to  the  systems  organization. 

(2)  What  information  about  the  system  it  wants 
to  include. 

(3)  Whether  "outside"  help  is  to  be  called  upon 
to  render  advice  and  assistance  in  preparing  the  proposal. 
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b.  The  user's  system  staff  (if  one  exists)  prepares 
the  proposal,  with  a  set  of  recommendations  as  to  priority, 
budget  allocation,  timing,  etc. 

c.  A  presentation  is  made  to  the  user's  management, 
who  decide  to  accept,  reject,  or  defer.   If  a  revision  is 
called  for  then  steps  b  and  c  above  are  repeated. 

d.  When  user's  management  accepts  the  proposal,  a 
formal  copy  is  forwarded  to  the  systems  organization  with  a 
request  for  further  action. 

3 .   Initial  User/System  Organizational  Assessment 

Determining  the  study  needs,  if  any,  to  convert  the 
proposal  into  a  formal  project-authorization  document  for 
final  management  action,  and  setting  up  a  study  team  to 
conduct  such  a  study. 

The  objectives  of  this  activity  are  to  determine 
whether  the  proposal  can  and  should  be  segmented  into  phases 
for  sequential  or  parallel  implementation,  to  determine  if 
the  phases  of  the  proposal  are  similar  in  scope  to  other 
planned  or  on-going  systems-development  activities,  and  to 
define  further  detailed  study  requirements  prior  to  recommend- 
ing project  authorization  (including  possibility  of  joint 
development  of  part  or  all  of  the  proposed  system  with  that 
of  other  users).   If  the  proposal  is  satisfactory  as  is,  and 
contains  adequate  information  in  the  form  necessary  for 
management  authorization,  the  next  activity  in  this  phase 
may  be  bypassed.   A  memorandum  specifying  that  the  proposal  is 
either  presently  adequate  for  management  authorization  purposes 
or  needs  further  study  should  be  produced. 
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Procedural  steps  for  this  activity  are  as  follows: 

a.  The  systems  organization  assigns  the  proposal 
assessment  responsibility  to  a  staff  group  where  user  and 
systems  personnel  establish  liaison  for  joint  assessment  of 
the  proposal. 

b.  For  systems  proposals  encompassing  more  than  one 
functional  area   an  attempt  should  be  made  to  segment  the 
proposal  into  a  number  of  modular  phases  which  could  be 
authorized  separately,  if  desired. 

c.  The  sequence  in  which  the  steps  should  be  under- 
taken and  completed  should  be  determined  based  on  logical 
precedence. 

d.  A  determination  is  made  of  the  possible  similarity 
in  scope  of  each  of  the  phases  to  other  proposed  or  on-going 
efforts. 

e.  The  requirements  for  further  study  of  those  phases 
requiring  early  management  authorization  is  determined, 
including  the  additional  information  to  be  developed. 

f.  Recommendations  are  developed  for  the  size, 
composition  and  work  plan  of  a  study  team. 

g.  A  study  team  manager  and  members  are  assigned  to 
begin  work  with  user  and  systems  organization  concurrence. 

4 .   Additional  Study 

Conducting  a  feasibility  study  and  preparing  a  feasi- 
bility report,  containing  recommendations  and  back-up  informa- 
tion for  management  authorization  of  a  project  or  series  of 
related  projects. 
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The  objectives  of  this  activity  are  identified  as: 

a.  To  identify  specific  phases  to  be  "pro jectized" 
initially. 

b.  To  develop   complete  data  on  the  project (s)  for 
management  approval. 

c.  To  view  proposed  projects  in  the  context  of  other 
systems  development  activities,  including:   determining 
whether  combining,  in  part  or  entirely,  with  similar  develop- 
ments is  feasible,  deciding  what  interfaces  must  be  provided 
with  other  systems,  and  ensuring  adaptability  of  the  proposed 
system  to  organization  change  and  growth. 

The  contents  of  a  Feasibility  Report  or  data  for 
management  consideration  and  project  guidance  is  outlined 
below  as  a  guide. 

a.  Description  of  the  overall  system  in  terms  under- 
standable to  management. 

b.  The  specific  scope  of  the  phase (s)  of  the  system 
for  which  approval  is  presently  being  requested. 

c.  Summary  of  findings,  conclusions. 

d.  Specific  recommendations. 

e.  Alternatives  considered,  approach  selected  for 
purposes  of  feasibility  evaluation. 

f.  Effect  of  selected  approach  on  operations  such 
as  people,  quality,  effectiveness,  cost  and  benefits  (by 
project  phases)  including  outlays  by  time  period,  savings 
(personnel  and  other) ,  present  value  and  discounted  cash  flow, 
intangible,  non-quantifiable  benefits  and  probability  of 
their  realization. 
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g.   Effect  on  existing  and  planned  systems,  and  what 
is  to  be  done  with  respect  to  those  systems. 

h.   Probability  of  technical  success  such  as  projec- 
tions of  technology  (state-of-the-art)  trends,  projections  of 
resource  availabilities,  comparison  of  requirements  with 
projections  (cost,  effectiveness,  schedule). 

i.   Recommended  plan  of  action: 

(1)  phases  to  be  approved  and  "projectized"  now. 

(2)  resources  required,  type  and  quantity  to  be 
assigned. 

(3)  further  study  required  prior  to  presentation 
of  further  phases  for  approval,  and  timing  of  the  necessary 
preliminary  studies. 

5 .   Management  Presentation 

A  presentation  leading  to  informed  understanding  of 
the  need  for  and  consequences  of  authorizing  the  project  in 
the  proposed  systems  area. 

The  goals  of  this  activity  are  as  follows: 

a.  To  assist  in  weighing  the  expected  payoff  of  the 
proposed  project  and  other  projects  competing  for  systems 
implementation  resources. 

b.  To  help  decide  when  and  at  what  level  of  effort 
a  project  should  be  established  in  order  to  maximize  the 
opportunity  for  significant  progress  without  significantly 
impeding  the  progress  of  other  important  efforts. 

c.  To  permit  consideration  of  payoff  opportunities 

in  terms  of  contribution  to  the  overall  division  or  organization 
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posture  in  systems  development,  and  not  merely  in  terms 
of  the  merits  of  a  project  as  an  isolated  system. 

d.  To  permit  cost/payoff  estimates  and  permit  evalua- 
tion, in  terms  of  management  objectives,  of  joint  development 
of  proposed  systems  among  more  than  one  division  or  functional 
group,  where  there  is  no  apparent  technical  or  functional 
reason  for  different  systems. 

e.  To  permit  consistency  in  the  evaluation  of  this 
project  against  other  proposed  projects  on  the  basis  of 
uniformly  complete  and  accurate  information. 

In  summary  the  Study  Team  should  present  its  findings 
and  make  its  recommendations  as  to  the  establishment  of  the 
project  and  a  proposed  work  plan  showing  scheduled  resource 
requirements.   The  planning  staff  should  present  its  analysis 
of  the  impact  of  proposed  project  on  resources  available  and 
on  other  systems  activities.   It  should  also  present  alterna- 
tive courses  of  action  realizing  that  management  may  request 
more  information  prior  to  making  a  decision,  or  may  take  under 
advisement  at  this  point  pending  a  decision. 
6 .   Management  Actions 

Project  approval  and  assignment  of  a  project  team  with 
project  responsibilities,  resource  levels,  etc.;  project 
disapproval  or  referral  for  more  study. 

The  target  aims  of  this  activity  are: 

a.   To  decide  whether  there  is  enough  information 
about  a  proposed  project  and  its  effects  to  make  an  intelligent 
allocation  of  resources. 
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b.  To  allocate  systems  resources  (principally 
personnel)  to  this  project,  as  compared  to  other  proposed 
projects  and  other  systems  activities  competing  for  them. 

c.  To  select  a  start  date  for  this  project. 

d.  To  assign  management  control  responsibility  for 
this  project  to  a  project  team. 

e.  To  establish  project  steering  responsibility  and 
reporting  frequency. 

f.  To  determine  benchmarks  or  checkpoints  to  be  met 
prior  to  the  approval  of  further  phases  of  the  project. 

g.  To  consider  and  make  policy  covering  the  general 
allocation  of  resources  among  projects,  and  between  projects 
and  non-project  activities. 

Payoff  information  may  be  based  on  no  more  than  an 
educated  guess  in  which  case  management  may  decide  that  further 
analysis  is  required  before  a  decision  as  to  priority  in  the 
use  of  resources  can  be  made,  especially  for  major  projects. 
Existing  projects  may  also  find  that  previously  assigned 
resources  are  inadequate,  or  that  schedules  must  be   altered. 
Requests  for  resource  changes  or  major  schedule  changes  must 
compete  for  resources  against  projects  being  newly  considered. 

If  further  study  prior  to  authorization  is  deemed 
necessary  then  the  study  team  is  notified  with  the  defined 
requirements  for  additional  information  and  a  due  date  is  set. 
Otherwise  a  project  and  a  project  team  is  established  to  start 
work  as  assigned  priority  dictates.   The  project  team  consists 
of  permanent  members  (including  a  project  manager)  drawn  from 
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the  user  organization,  systems  departments,  and  other  groups 
as  needed,  and  "loaned"  to  serve  on  the  team  for  the  duration. 

In  some  cases  further  study  on  existing  projects  may 
be  deemed  necessary  before  future  phases  are  authorized. 
This  would  be  true  particularly  if  the  scope  of  the  original 
study  did  not  carry  through  all  phases  to  project  completion, 
or  if  problems  arose  in  the  course  of  the  project  such  that 
certain  previously  arrived  at  conclusions  were  made  invalid. 

B.   PHASE  TWO  -  DEVELOPMENT 

Once  the  scope  and  general  configuration  of  the  MIS  have 
been  established,  the  detailed  design  of  the  system  may  be 
started.   The  first  step  in  systems  design  is  not  a  technical 
one.   It  is  concerned  with  gaining  support  for  the  work  that 
follows.   Systems  designers  must  have  the  support  of  most 
members  of  the  organization  in  order  to  obtain  acceptance 
of  the  final  system.   At  a  minimum,  members  of  the  organization 
should  be  informed  of  the  objectives  and  nature  of  the  study. 
It  is  preferable,  if  possible,  to  draw  many  members  into  the 
study,  at  least  in  some  small  way. 

The  aim  of  the  detailed  design  is  to  furnish  a  description 
of  a  system  that  achieves  the  goals  of  the  gross  system  design 
requirements  arrived  at  during  the  feasibility  (gross)  design. 
This  description  consists  of  drawings,  flowcharts,  hardware 
equipment  requirements  (computers,  peripherals,  communications, 
terminals),  programming  languages  to  be  used,  procedures, 
support  tasks,  specification  of  information  record  and  file 
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designs   (input,  output,  files,  tables,  etc.)  and  organization 
and  operating  manuals  required  to  run  the  system.   Also  part 
of  the  design  is  the  documentation  of  analysis  and  testing, 
which  justifies  the  design.   The  design  must  be  sufficiently 
detailed  that  operating  management  and  personnel  may  implement 
the  system.   Whereas  the  gross  design  gives  the  overall  per- 
formance specifications  for  the  MIS,  the  detailed  design 
yields  the   construction  and  operating  specifications. 
1 .   Define  the  Subsystems 

Although  the  gross  design  requires  some  assumptions 
concerning  the  subsystems,  it  is  necessary  now  to  review  these 
subsystems  and  to  redefine  them  if  it  seems  appropriate. 
Based  upon  the  gross  design,  investigation  of  the  detailed 
activities  of  each  major  activity  must  be  undertaken.   Each 
large  system  must  be  broken  down  to  determine  all  activities 
required  and  the  necessary  information  inputs  and  outputs  of 
each  activity. 

The  information  system  must  be  based  upon  the  operat- 
ing system.   Once  this  operating  system  is  outlined  by  the 
selection  of  a  gross  concept,  certain  basic  relationships 
among  major  activities  become  more  or  less  fixed.   However, 
there  is  still  considerable  freedom  in  establishing  the 
detailed  activities  and  their  relationships.   The  degree  of 
breakdown  of  the  major  activities,  of  course,  determines  the 
size  and  complexity  of  the  network.   If  the  activities  are 
broken  down  too  finely,  the  design  will  never  be  completed. 
If  a  major  activity  is  broken  down  too  coarsely,  vital 
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material,  information,  and  decision  needs  will  not  be  factored 
into  the  design.   Furthermore,  optional  rearrangement  or 
regrouping  of  activities  will  not  be  examined. 
2 .   Operations  and  Information  Flows 

The  development  of  the  detailed  design  is  first 
carried  out  for  the  subsystem,  functional,  and  task  levels 
of  detail.   It  is  very  similar  to  detailed  engineering  design, 
which  requires  trial  and  error,  shifting  operations  to  find 
good  arrangements,  and  performing  calculations  to  check  out 
the  system.   The  equivalents  of  engineering  sketches  in  MIS 
design  are  the  flowcharts.   There  are  three  types  of  systems 
flowcharts  [Refs.  8,  23,  and  24]: 

a.  Task-oriented  charts.   These  are  block  diagrams 
showing  the  relationships  among  the  various  tasks  or  activi- 
ties.  Subsequently,  the  detailed  elemental  steps  required  to 
complete  an  activity  are  analyzed  and  described  step  by  step 
on  an  operations  analysis  form  (sometimes  called  a  flow- 
process  chart) . 

b.  Forms-oriented  charts.   These  charts  identify  the 
forms  used  in  communicating  or  reporting  and  trace  the  flow 
of  all  copies  through  the  organization.   In  some  cases,  the 
chronological  movement  may  receive  emphasis. 

c.  Program  flowcharts  (block  diagrams).   Prepared 
by  the  people  who  give  instructions  to  the  computer,  the 
program  flowchart  is  a  fundamental  tool  of  programming, 
designed  to  show  the  logical  sequence  of  steps  to  be  carried 
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out  by  the  computer.   It  structures  logic  that  the  coding 
of  the  programs  will  follow. 

The  flowcharts  are  not  the  complete  detailed  design. 
They  show  primarily  flows  and  relationships.   Inputs  and  out- 
puts are  shown  only  in  gross  form.   The  quantitative  relations 
among  elements  in  the  systems  must  be  expressed  in  terms  of 
mathematical  models.   Where  this  is  not  possible,  detailed 
verbal  descriptions  must  be  used  to  actually  develop  the 
detailed  operating  design.   The  flowcharts  are  important, 
however,  in  developing  the  information  necessary  for  mana- 
gerial decisions  with  respect  to  the  design  for  model  con- 
structions, and  for  programmed  decision  making  in  system 
operation . 

3 .  Determine  Degree  of  Automation 

Each  operation  in  the  flowcharts  should  next  be 
examined  to  establish  the  level  of  automation  possible.   By 
listing  each  operation  along  the  horizontal  axis  of  a  chart 
and  levels  of  automation  along  the  vertical  axis,  an  "auto- 
mation profile"  may  be  plotted.   Widely  contrasting  levels  of 
automation  in  a  system  may  be  suspect  and  should  be  examined. 

4 .  Develop  the  Data  Base 

The  data  base  is  the  data  that  must  be  obtained  and 
usually  stored  for  later  retrieval  for  managerial  decision 
making.   It  also  consists  of  data  that  will  be  utilized  in 
programmed  decision  making  and  real-time  control.   The  data 
base  is  derived  from  the  needs  of  management  for  information 
to  guide  the  total  organizational  system. 
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One  of  the  important  characteristics  of  data  bases  is 
that  they  can  be  accessed  by  one  or  more  information  systems 
and/or  one  or  more  organizational  units.  Thus  input  errors  may 
be  introduced  by  many  different  input  sources;  fixing  the 
accountability  for  them  becomes  a  much  more  difficult  task. 
In  addition  the  confidential  nature  of  certain  data  files 
demands  that  data  base  access  be  limited  to  individuals  who 
have  a  demonstrated  "need  to  know"  [Ref.  23]. 
5 .   Develop  the  Software 

Although  software  programming  development  in  the 
technical  sense  is  not  a  primary  concern  of  management, 
management  does  have  the  responsibility  of  insuring  that  the 
software  is  an  economical  and  effective  part  of  the  MIS. 
Software  development,  particularly  good  programming,  is 
generally  an  expensive  activity  that  cannot  be  slighted. 

The  coordination  of  the  systems  design  group  and  the 
computer  organization  should  start  at  the  time  of  the  gross 
design.   Trained  programmers  should  be  on  hand  at  the  start 
of  detailed  design  work  and  many  months  prior  to  installation. 
There  are  some  principal  steps  in  softward  development  for 
systems  over  which  management,  through  the  systems  designers, 
should  maintain  surveillance.   These  steps  carried  out  by  the 
computer  organization,  are: 

a.  Develop  standards  and  procedures  for  programming. 
Standardized  charting  symbols,  techniques,  and  records  should 
be  maintained. 
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b.  Study  the  gross  system  specifications  and  work 
with  the  system  designers  in  the  development  of  the  detailed 
design.   The  computer  programmers  should  be  a  part  of  the 
design  team  by  contributing  their  expertise  as  needed. 

c.  Develop  the  data-processing  logic  and  prepare 
the  programming  flowcharts.   When  the  programming  charts  are 
completed,  they  should  be  reviewed  by  the  systems  design 
group. 

d.  Code  the  instructions  given  by  the  flowcharts. 
This  is  the  writing  of  detailed  instructions  to  the  computer. 
Good  coding  should  balance  gains  from  economical  use  of 
machine  operation.   Another  important  goal  for  the  coding 
process  is  to  build  error  control  into  the  machine  instructions. 

e.  Test  the  program.   The  aim  is  to  find,  diagnose, 
and  correct  errors  by  running  sample  problems  and  checkout 
programs  on  the  computer.   Actually  this  "debugging"  process 
often  continues  into  the  implementation  phase,  where  it  is  a 
much  more  expensive  process. 

f.  Document  the  programming,  coding,  and  testing. 
This  is  an  extremely  important  step.   Too  often  rough  sketches, 
preliminary  programs  and  codes,  and  test  results  are  not  up- 
dated to  the  "final"  or  most  recent  status.   Not  only  should 
documentation  be  maintained  completely  up  to  date,  but  the 
contents  should  be  easily  interpreted  by  anyone  skilled  in 

the  field.   It  is  the  management's  responsibility  to  insure 
that  this  proper  documentation  takes  place. 
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6 .  Information  Outputs 

A  system  of  reports  should  be  established,  not  to 
isolate  the  manager  from  routine  detail  but  to  provide  him 
with  increasing  detail  at  each  level  of  operation  as  he  needs 
it  to  solve  problems  and  make  decisions.   Standard  typed 
reports  and  well-planned  computer-output  summary  reports 
will  probably  be  the  basic  formats  for  communication  of 
information  to  managers  for  some  time  yet.   Video  communica- 
tions and  cathode-ray-type  presentation  of  information  offer 
speed  and  flexibility. 

The  growing  computer  sophistication  of  today's 
managers  is  increasing  the  use  of  time-sharing  terminals  as 
a  means  of  getting  information  to  managers.   Managers  are 
able  to  utilize  models  to  ask  the  "What  if  I  do  this...?" 
type  of  question  and  receive  the  information  within  seconds 
or  minutes. 

In  general,  the  format  should  be  established  to  save 
the  manager's  time.   A  wide  variety  of  new  communications  and 
display  equipment  has  been  developed  and  the  systems  designer 
should  remain  abreast  of  these  developments. 

7 .  Document  the  Detailed  Design 

The  end  product  of  the  detailed  design  project  is 
production  of  the  documents  that  specify  the  system,  its 
operation,  and  its  design  justification.   Documentation 
should  consist  of: 

a.  A  summary  flowchart. 

b.  Detailed  flowcharts. 
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c.  Operations  activity  sheets  showing  inputs, 
outputs,  and  transfer  functions. 

d.  Specification  of  the  data  base  or  master  file. 

e.  Computer  hardware  requirements. 

f.  Software  (programs). 

g.  Personnel  requirements  by  type  of  skill  or 
discipline. 

h.  Final  (updated)  performance  specifications. 

i.  Cost  of  installation  and  implementation  of  the 


system. 


j.   Cost  of  operating  the  system  per  unit  of  time, 
k.   Program  for  modification  or  termination  of  the 


system. 


1.   An  executive  digest  of  the  MIS  design.   This  is 
a  report  that  top  management  can  read  rapidly  in  order  to  get 
the  essence  of  the  system,  its  potential  for  the  organization, 
its  cost,  and  its  general  configuration. 

Some  documentation  should  be  on  standardized  forms. 
Input-output-activity  diagrams  or  listings  are  an  example. 
Obviously,  standard  symbols  should  be  used  on  flowcharts  and 
guidelines  should  be  established  for  flowchart  format.   Some 
documentation  is  unique  to  a  project,  such  as  the  data  base, 
and  the  format  and  classification  of  items  should  be  deter- 
mined by  the  needs  of  the  particular  user.   Other  documenta- 
tion should  simply  follow  good  reporting  style. 


C.   PHASE  THREE  -  IMPLEMENTATION 

The  three  main  phases  in  implementation  take  place  in 
series:   these  are  the  initial  installation,  the  test  of  the 
system  as  a  whole  and  the  evaluation  of  the  system.   On  the 
other  hand,  many  implementation  activities  should  be  under- 
taken in  parallel  in  order  to  reduce  implementation  time. 
For  example,  acquisition  of  data  for  the  data  base  and  forms 
design  for  collection  of  information  may  be  carried  out  in 
parallel.   Training  of  personnel  and  preparation  of  software 
may  be  in  parallel  with  each  other  and  with  other  implementa- 
tion activities. 

It  is  apparent,  then,  that  the  first  step  in  the  imple- 
mentation procedure  is  to  plan  the  implementation. 

1 .   Implementation  Alternatives 

There  are  four  basic  methods  for  implementing  the  MIS 
once   work  has  been  completed.   These  are: 

a.  Install  a  system  in  a  new  operation  or  organiz- 
tion,  one  just  being  formed. 

b.  Cut  off  the  old  system  and  install  the  new.   This 
produces  a  time  gap  during  which  no  system  is  in  operation. 
It  is  practical  only  for  small  systems  where  installation 
requires  one  or  two  days.   An  exception  to  this  would  be  the 
installation  of  a  larger  system  during  an  organization's 
vacation  shutdown  or  some  other  period  of  inactivity. 

c.  Phase-in  by  segments.   Small  parts  or  subsystems 
are  substituted  for  the  old.   If  this  method  is  possible, 
some  careful  questions  should  be  asked  about  the  design  of 
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the  new  system.   Is  it  really  just  an  automation  of  isolated 
groups  of  clerical  activities?   Generally,  new  systems  are  not 
substitutable  piece  by  piece  for  previous  nonsystems.   However, 
in  upgrading  old  systems,  this  may  be  a  very  desirable  method. 

d.   Operate  in  parallel  and  phase-in.   The  new  system 
is  installed  and  operated  in  parallel  with  the  current  system 
until  it  has  been  checked  out;  then  the  current  system  is 
cut  out.   This  method  is  expensive  because  of  the  manpower 
and  related  costs.   However,  it  is  required  in  certain  essen- 
tial systems.   Its  big  advantage  is  that  the  system  is  fairly 
well  debugged  when  it  becomes  the  essential  information 
system  of  the  organization. 

2 .   Obtain  Space,  Plan  Layout 

The  installation  of  a  new  system  to  replace  a  current 
one  may  require  a  major  revision  of  facilities  as  well  as 
completely  new  office,  computer  room,  and  production  layouts. 
The  MIS  project  manager  must  prepare  rough  layouts  and  esti- 
mates of  particular  floor  areas  he  feels  will  be  needed.   He 
should  then  prepare  cost  estimates  and  submit  a  proposal  for 
management's  approval. 

Facilities  and  space  planning  should  begin  as  soon 
as  approval  of  gross  space  allocations  has  been  obtained. 
The  urgency  for  such  planning  is  twofold.   First,  there  may 
be  a  long  lead  time  if  new  partitions,  electrical  work,  air- 
conditioning,  or  even  new  buildings  are  required.   Second, 
the  detailed  work  flow  depends  upon  the  physical  arrangements 
of  the  buildings.   The  training  of  operations  personnel  will 
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be  more  successful  if  it  is  based  on  exact  physical  relation- 
ships among  the  people  and  the  equipment. 

Space  planning  must  take  into  account  the  space 
occupied  by  people,  the  space  occupied  by  equipment,  and  the 
movement  of  people  and  equipment  in  the  work  process. 
Related  to  these  are  the  number  and  kinds  of  exits;  storage 
areas;  location  of  utilities,  outlets,  and  controls;  environ- 
mental requirements  for  the  equipments;  safety  factors;  and 
working  conditions  for  the  personnel.   It  is  a  short-sighted 
policy  to  scrimp  on  facilities  and  human  environment  when  a 
major  renovation  is  required  to  install  a  new  system. 
3 .   Develop  Procedures  for  Implementation 

Procedures  for  evaluating  and  selecting  hardware  must 
be  spelled  out.   Procedures  for  buying  or  constructing  soft- 
ware should  be  established.   Procedures  for  phasing  in  parts 
of  the  MIS  or  for  operating  the  MIS  in  parallel  must  be 
developed.   Obviously  there  are  many  procedures  that  must  be 
delineated  in  advance  if  the  entire  implementation  is  to  be 
saved  from  chaos. 

A  major  part  of  implementing  the  MIS  is  the  testing 
of  each  segment  of  the  total  system  as  it  is  installed.  So 
far,  the  only  testing  that  has  been  done  is  a  simulation  of 
the  system  during  the  detailed  design  phase.  The  testing  of 
segments  of  MIS  during  installation  requires  application  of 
line  personnel  to  actual  files,  software,  and  hardware  for 
operations  or  specially  designed  test  problems. 
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It  is  necessary  to  develop  the  testing  procedures  on 
the  basis  of  the  design  and  test  specifications.  The  proce- 
dures should  prescribe: 

a.  Which  segments  of  the  system  will  be  tested 

b.  When  such  tests  are  to  be  performed 

c.  Test  problems  to  be  run 

d.  Who  will  perform  the  tests 

e.  How  the  tests  will  be  run 

f.  Who  will  evaluate  test  results  and  approve  the 
system  segment  or  recommend  modification. 

For  example,  the  complete  detailed  procedure  for  the  accomp- 
lishment of  the  test  specification  might  include  organization 
of  personnel  for  conduct  of  the  test;  provision  of  necessary 
forms  and  data  sheets;  statement  of  conditions  to  exist  at 
the  start  of  the  test;  a  list  of  all  equipment,  software,  and 
file  data  required  for  the  test;  and  step-by-step  procedure 
for  all  the  people  participating  in  the  test. 

Components  may  be  tested  relatively  independently 
of  the  system  to  which  they  belong.   Test  for  accuracy,  range 
of  inputs,  frequency  of  inputs,  usual  operating  conditions, 
human  factor  characteristics,  and  reliability  are  all  of 
concern.   As  more  components  are  installed,  subsystems  may 
be  tested.   There  is  a  considerable  difference  between  the 
testing  of  a  component  and  the  testing  of  a  system.   Systems 
tests  require  verification  of  multiple  inputs,  complex  logic 
systems,  interaction  of  humans  and  widely  varied  equipment, 
interfacing  of  systems,  and  timing  aspects  of  the  many  parts. 
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If,  for  example,  the  programming  for  the  computer  fails  to 
work  in  the  system  test,  costly  delays  may  take  place.   Often, 
minor  difficulties  cropping  up  require  redesign  of  forms, 
procedures,  work  flow  or  organizational  changes.   The  training 
program  itself  is  being  tested,  since,  if  the  supervisors  and 
operators  lose  confidence  in  the  system  at  this  point,  they 
may  resist  further  implementation  of  the  new  system  in  subtle 
ways . 

4 .   Train  the  Personnel 

A  program  should  be  developed  to  impress  upon  manage- 
ment and  support  personnel  the  nature  and  goals  of  the  MIS 
and  to  train  operating  personnel  in  their  new  duties. 
Particular  attention  should  be  paid  to  the  training  of  first- 
line  supervisors.   They  must  have  a  thorough  understanding  of 
what  the  new  MIS  is  like  and  what  it  is  supposed  to  do. 
Since,  in  essence,  they  oversee  the  operation  of  the  system, 
they  must  learn  how  it  will  operate.   They  are  faced  with 
many  changes  in  their  work  and  they  must  obtain  acceptance 
of  changes  by  their  subordinates. 

Finally,  longer  and  more  formal  training  programs 
should  be  established  for  people  who  perform  the  daily  opera- 
tional tasks  of  the  MIS.   These  are  the  clerks,  the  computer 
operators,  the  input  and  output  machine  operators,  file 
maintenance  personnel,  and  possibly  printing  production  and 
graphic  arts  personnel. 
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5 .  Develop  the  Software,  Acquire  the  Hardware 

A  comprehensive  discussion  of  the  preparation  of 
computer  programs  and  the  evaluation  of  computer  and  peripheral 
equipment  does  not  fall  within  the  constraints  of  this  thesis 
effort,  rather  with  identifying  the  managerial  considerations 
of  MIS  design.   Systems  designers  and  programmers  provide  the 
flow  diagrams  and  the  block  diagrams  during  the  development 
stage.   Some  modification  may  be  required,  however,  as  the 
implementation  stage  progresses.   In  the  implementation  stage, 
coders  convert  block  diagrams  into  sequences  of  statement  or 
instructions  for  the  processing  (computer)  equipment. 

The  development  of  software  and  the  acquisition  of 
new  equipment  are  usually  the  limiting  items  in  getting  an 
MIS  implemented  [Ref.  8].   When  possible,  these  tasks  should 
be  started  during  the  design  stage.   There  is,  of  course, 
some  risk  of  loss  in  starting  early,  but  it  must  be  balanced 
against  the  considerable  delay  involved  in  the  sequential 
approach  to  design  and  implementation  of  the  MIS. 

6 .  Develop  Forms 

A  vast  amount  of  detailed  data,  both  external  and 
internal  to  the  organization,  must  be  collected  for  input  to 
the  MIS.   Obviously,  the  form  insures  that  the  right  informa- 
tion is  supplied  in  a  manner  that  simplifies  processing  for 
computer  storage.   Many  factors  affect  the  design  of  both 
input  and  output  forms.   When  considering  a  new  form  the 
first  questions  should  always  be: 
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a.  Is  this  form  really  necessary? 

b.  What  form(s),  if  any,  will  it  replace? 

c.  Can  existing  forms  be  revised  to  include  the 
required  information? 

d.  How  was  this  information  previously  supplied? 

After  gathering  satisfactory  answers  to  these  ques- 
tions then  the  design  of  the  new  form  can  proceed.   The  most 
important  principle  of  form  design  is  to  plan  the  form  with 
the  user(s)  in  mind.   Other  considerations  should  be: 

a.  How  many  copies  are  to  be  prepared? 

b.  Will  the  form  be  permanent? 

c.  Is  it  for  internal  or  external  use? 

d.  What  quality  of  paper  and  size  of  form  should  be 
used? 

e.  Is  the  form  simple  and  easy  to  understand? 

f.  Is  the  make-up  of  the  form  straight-forward  and 
in  accordance  with  machine  processing  acceptance? 

The  following  principles  should  contribute  to  good 
form  design: 

a.  Bold  type  should  be  used  to  emphasize  important 
information. 

b.  Filing  information  should  be  near  the  top  of  the 
form. 

c.  Every  form  should  have  a  title. 

d.  Headings  should  be  as  small  as  possible,  leaving 
sufficient  space  for  written  data. 
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e.  A  good  printing  style  should  be  selected  to  make 
the  form  attractive  in  appearance. 

f.  The  form  should  include  only  essential  information. 

g.  The  form  should  be  designed  so  that  a  minimum  of 
recording  and  recopying  is  required. 

h.   If  the  form  precedes  another  form,  or  is  dependent 
on  another  form,  the  same  general  sequence  and  arrangement 
should  be  followed  so  that  recopying  and  recording  can  easily 
be  accomplished. 

i.  Once  the  form  is  designed,  it  should  be  analyzed 
to  determine  whether  it  is  sufficiently  clear  and  all  neces- 
sary instructions  are  printed  on  the  form. 

Output  forms  of  the  MIS  must  be  prepared  at  the 
implementation  stage,  when  they  can  be  both  designed  and  tested, 
Further,  the  problems  of  printing  and  inventory  size  and  loca- 
tion must  be  resolved.   The  output  forms  are  what  the  managers 
see,  and  so  these  forms  or  formats  should  be  designed  so  that 
key  information  and  variances  are  easily  discernible.   A 
periodic  report  form  should  be  a  summary  form  that  is  keyed 
to  a  hierarchy  of  increasingly  detailed  formats  or  forms. 
Managers  may  then  pursue  specific  questions  easily  by  asking 
for  the  underlying  details. 
7 .   Develop  the  Files 

In  the  development  phase,  each  item  of  data  for  the 
files  is  specified  and  the  retrieval  methods  (indexes)  are 
developed.   In  the  implementation  phase,  forms  must  be 
designed  so  that  the  data  may  be  analyzed  by  the  programmers 
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and  coders  for  storage  in  the  computer.   Thus,  the  file  name, 
maximum  number  of  characters  required  to  record  each  data 
element,  frequency  of  access,  volume  of  operations  on  the 
element,  retention  characteristics,  and  updating  frequency 
are  examples  of  relevant  information  required  to  translate 
a  specification  into  a  file  element  [Ref.  23].   The  develop- 
ment of  files  or  data  bases  belongs  in  the  conceptual  realm 
of  information  system  designers  and  storage  and  retrieval 
experts.   The  translation  of  specifications  for  files  into 
computer  programs  is  a  function  of  computer  specialists. 

8 .  Cut  Over 

Cutover  is  the  point  at  which  the  new  component 
replaces  the  old  component  or  the  new  system  replaces  the  old 
system.   This  usually  involves  a  good  deal  of  last  minute 
physical  transfer  of  files,  rearrangement  of  office  furniture, 
and  movement  of  work  stations  and  people.   Old  forms,  old 
files,  and  old  equipment  are  suddenly  retired. 

Despite  component  and  system  testing,  there  are  likely 
to  be  "bugs"  in  the  system.   Having  extra  supervisory  help, 
with  the  systems  designers  on  hand,  is  one  way  of  preventing 
first-day  cutover  panic.   Design  analysts  should  also  be 
present  to  iron  out  "bugs"  of  all  kinds  that  may  arise. 

9 .  Document  the  System 

Documentation  of  the  MIS  means  preparation  of  written 
descriptions  of  the  scope,  purpose,  information  flow  compo- 
nents, and  operating  procedures  of  the  system.   Documentation 
is  not  a  frill;  it  is  a  necessity  for  trouble-shooting, 
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replacement  of  subsystems,  interfacing  with  other  systems, 
and  for  training  new  operating  personnel,  and  also  for 
evaluating  and  upgrading  the  system. 

If  the  system  is  properly  documented: 

a.  A  new  team  of  operators  could  be  brought  in  and 
could  learn  to  operate  the  MIS  on  the  basis  of  the  documenta- 
tion available. 

b.  Designers  not  familiar  with  the  organization  or 
MIS  could,  from  the  documentation,  reconstruct  the  system. 

c.  A  common  reference  design  is  available  for 
managers,  designers  and  programmers  concerned  with  system 
maintenance. 

d.  The  information  systems  analyst  will  have  a 
valuable  data  source  for  developing  new  MIS,  schedules, 
manpower  plans,  and  costs. 

D.   PHASE  FOUR  -  UTILIZATION 

The  Use  period  of  the  System  Life  Cycle  is  that  long 
period  where  the  system  can  now  be  operated  to  fulfill  its 
system  requirements.   Once  the  new  system  is  in  operation, 
system  evaluation  and  modification  begin.   This  phase  should 
be  a  continuing  effort  which  seeks  to  take  advantage  of  new 
developments  as  they  occur.   It  is  during  this  period  that 
the  true  cost-effectiveness  of  the  system  can  be  measured. 
The  Use  period  really  includes  three  activities,  Operations 
and  Support,  Modification,  and  Retirement. 
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Systems  design  involvement  in  the  system  is  not  complete 
until  the  system  is  obsolete  and  finally  retired  from  use. 
During  the  Use  period,  some  problems  with  the  system  not 
previously  encountered  will  arise.   These  serve  as  a  basis 
for  design  changes.   In  addition,  new  uses  or  requirements 
for  the  system  will  result  in  modifications  to  meet  changing 
requirements.   In  this  way,  early  obsolescence  is  minimized. 

Finally,  when  the  system  no  longer  proves  to  be  cost- 
effectively  used  or  modified  to  meet  existing  or  new  system 
requirements,  it  is  retired.   This  will  usually  generate  new 
system  requirements  and  the  System  Life  Cycle  will  start  all 
over  again.   Sometimes,  the  System  Life  Cycle  starts  with  a 
brand  new  requirement  rather  than  as  a  second-generation 
system.   This  may  be  as  a  result  of  a  new  technological  break- 
through which  allows  us  to  feasibly  and  effectively  do  what 
we  could  not  previously. 
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V.   INFORMATION  SYSTEM  ANALYSIS 

A.   SOTAP  IDENTIFIABLE  ELEMENTS 

Certain  elements  of  an  information  flow  system  have 
already  been  tentatively  identified  by  SOTAP  [Ref .  25] : 

1.  Training  and  Assessment  Data/Scoring  Information 
Sheets  for  data  transfer  to  a  storage  and  analysis  facility 
will  be  in  the  form  suitable  for  reading  by  an  optical 
scanning  device  such  as  used  by  PTEP.   Appendix  C  is  a 
copy  of  the  current  PTEP  scoring  form. 

2.  Use  of  in-place  PTEP  and  its  on  site  support 
personnel  for  the  actual  handling  of  assessment  and 
training  information  if  PTEP  is  used  for  the  formation 
of  a  SOTAP  IFAS. 

3.  Periodic,  NUSC  sponsored  operational  training 
meetings  for  SOT  instructors.   These  meetings  will  facili- 
tate a  free  flow  of  information  between  the  SOT  training 
sites  in  New  London,  Connecticut  and  Charleston,  South 
Carolina. 

4.  Navy  sponsored  pre/post  SOT  training  conferences 
which  will  aid  in  establishing  the  training  syllabus  for 
a  particular  sonar  team  as  it  begins  its  week  of  SOT 
training  or  deployed  shipboard  training. 

5.  A  data  storage  and  analysis  capability  will  be 
provided  under  SOTAP. 
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B.  PTEP  BACKGROUND 

OPNAV  INSTRUCTION  1500. 23A,  of  15  June  1972,  established 
the  Fleet  Ballistic  Missile  Weapon  System  Training  Program 
along  with  its  major  elements,  one  of  which  is  the  Personnel 
and  Training  Evaluation  Program  (PTEP) .   The  administration 
of  PTEP  tasks  (encompassing  personnel  testing,  data  collec- 
tion, analysis  and  evaluation,  and  EDP  support)  are  conducted 
and  controlled  in  an  organized  and  standardized  manner  to 
ensure  the  continuity  and  reliability  of  required  input  data 
to  PTEP  and  the  validity  and  relevancy  of  PTEP  feedback 
information  (trends,  deficiencies,  and  recommendations)  to 
other  Training  Program  activities  and  commands.   As  estab- 
lished by  OPNAV  NOTICE  5450,  of  19  February  1974,  authority 
and  responsibility  for  Polaris/Poseidon  PTEP  implementation 
are  delegated  by  the  Chief  of  Naval  Operations  to  the  Chief 
of  Naval  Education  and  Training,  and  are  exercised  by  the 
Chief  of  Naval  Technical  Training  through  the  Central  Test 
Site  (CTS)  for  PTEP.   CTS  directs  the  CTS  Detachments  in 
administering  PTEP  and  conducting  evaluations.   CTS  is 
located  at  the  Dam  Neck  training  site.   CTS  Detachments  are 
located  at  the  Charleston,  New  London,  and  Pearl  Harbor 
training  sites.   Figure  V-l  shows  the  PTEP  organization. 

C.  PTEP  DEFINITION  AND  RESPONSIBILITIES 

PTEP  serves  as  the  evaluation  element  of  the  Training 
Program.   It  provides  the  organization,  procedures,  and 
responsibilities  for  the  qualitative  assessment  of  the 
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technical  proficiency  of  personnel,  and  the  evaluation  of 
the  effectiveness  of  all  Training  Program  elements  in 
defining  and  providing  efficient  training,  and  the  reporting 
of  findings  and  formulated  corrective  action  recommendations, 

The  measurement  of  personnel  proficiency  is  accomplished 
through  the  administration  of  standardized  tests  which  are 
based  on  the  personnel  knowledge  and  skill  requirements  set 
forth  in  the  Personnel  Performance  Profiles  (PPP)  and  the 
Training  Path  System  (TPS) ,  both  of  which  are  elements  of 
the  Training  Program.   Personnel  test  results  are  analyzed 
and  evaluated,  in  conjunction  with  other  supportive  data, 
to  identify  trends  and  deficiencies. 

Training  effectiveness  is  assessed  through  the  indi- 
vidual and  collective  evaluation  of  all  elements  of  the 
Training  Program.   Training  materials  are  analyzed  and 
evaluated,  in  conjunction  with  other  pertinent  data  (e.g., 
criteria  on  which  the  training  is  based  and  personnel  test 
results)  to  identify  trends  and  deficiencies. 

Identified  trends  and  deficiencies  are  studied  to 
determine  causes  within  the  Training  Program;  and  positive 
recommendations  for  corrective  actions  are  formulated. 
These  findings  and  recommendations  are  reported  to  appro- 
priate commands  for  use  as  the  basis  for  implementing 
improvements  in  training  and  in  all  Training  Program  ele- 
ments, and  to  assist  in  planning  training  and  in  determining 
the  most  effective  use  of  personnel. 
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Administration  of  the  fully  implemented  PTEP  occurs  in 
an  iterative  cycle  consisting  of  data  collection,  analysis 
and  evaluation,  and  reporting.   The  primary  component  of 
PTEP  is  analysis  and  evaluation.   All  other  PTEP  tasks 
serve  in  supportive  roles,  either  providing  data  input  to 
the  analysis  and  evaluation  effort,  providing  data  processing 
support,  or  providing  documentation  of  the  procedures  for 
analysis  and  evaluation  and  the  other,  supportive  PTEP 
components. 

Knowledge  and  skill  test  instruments  are  designed  for 
PTEP  to  measure  specific  achievement  levels  delineated  by 
the  PPP  and  TPS.   The  administration  of  selected  groups  of 
test  instruments  assist  in  the  identification  of  trends  and 
deficiencies  in  personnel  proficiency  and  training  effec- 
tiveness related  to  specific  PPP  and  TPS  knowledge  and  skill 
requirements . 

The  primary  types  of  test  instruments  used  in  Polaris/ 
Poseidon  PTEP  tests  are  multiple-choice  knowledge  test 
items  and  simulated  skill  test  exercises.   The  acquisition 
of  the  test  instruments  is  based  on  the  requirements  defined 
when  the  specific  personnel  testing  objectives  (quantitative 
and  qualitative)  are  determined.   Test  instrument  require- 
ments are  defined  in  terms  of  the  detailed  components  of 
the  PPP  and  TPS;  and,  thus,  they  provide  for  complete  accounta- 
bility regarding  the  capability  of  PTEP  personnel  testing 
and  its  extent  of  coverage.   Test  instruments  are  obtained 
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from  training  system  contractors,  Navy  training  activities, 
CTS,  and  the  CTS  Detachments.   Review  and  maintenance  (for 
format,  currency,  effectiveness,  and  relevancy)  of  the  test 
instruments  is  an  on-going  task. 

Knowledge  test  items  are  either  open-book  or  closed-book 
type,  depending  on  the  specific  testing  objectives  and  the 
operational  requirements.   Test  items  are  prepared  in 
accordance  with  the  specifications  set  forth  in  NAVORD  OD 
45519  to  ensure  the  use  of  standardized  format  and  conformance 
to  the  PPP  and  TPS.   Upon  receipt,  CTS  personnel  review 
test  items  for  technical  accuracy,  relevancy,  and  conformance 
to  the  prescribed  specifications  and  test  instrument  require- 
ments.  The  test  items  are  then  input  to  the  EDP  file  of 
test  items,  from  where  they  are  selected  for  use  in  PTEP 
personnel  tests. 

Skill  test  exercises  are  equipment  simulation  testing 
devices.   These  exercises  are  provided  to  CTS  in  manuscript 
form,  from  which  they  are  verified  for  specified  applica- 
bility with  respect  to  the  PPP  and  TPS  and  for  technical 
accuracy  and  relevancy  by  CTS  for  administration  in  PTEP 
skill  test  parts. 

Personnel  testing  is  the  component  of  PTEP  which  provides 
the  primary  source  of  data  required  to  determine  individual 
proficiency  levels,  with  respect  to  knowledge  and  skill 
achievement,  and  to  determine  training  effectiveness.   Testing 
is  accomplished  through  the  administration  of  standardized 
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tests  to  personnel  whose  training  is  provided  by  the  Training 
Program.   Test  results  are  reported  to  the  appropriate  commands 
to  assist  in  planning  training  and  in  determining  the  most 
effective  use  of  personnel,  and  are  input  to  the  analysis 
and  evaluation  component  of  PTEP  to  assist  in  identifying 
and  verifying  trends  and  deficiencies,  and  to  support  the 
formulation  of  recommendations  to  increase  the  effective- 
ness of  the  Training  Program.   Two  types  of  tests  are  pri- 
marily used  in  Polaris/Poseidon  PTEP:   System  Achievement 
Tests  (SATs)  and  Course  Achievement  Tests  (CATs) .   Particu- 
lar test  versions  are  comprised  of  knowledge  test  items 
and/or  skill  test  devices,  depending  on  their  availability 
and  the  testing  objectives. 

SATs  are  used  to  measure  personnel  proficiency,  relative 
to  the  overall  knowledge  and  skill  requirements  defined  in 
the  PPP  and  TPS  for  specific  personnel  categories  Navy 
Enlisted  Classifications  (NECs) ,  thereby  determining  the 
adequacy  of  personnel  in  supporting  the  mission.   Each  SAT 
is  applicable  to  a  particular  Training  Path  Chart  (TPC) , 
and  consists  of  knowledge  test  items  and/or  skill  test  devices 
which  sample  from  among  all  of  the  Training  Objective  State- 
ment (TOS)  knowledge  depths  and  skill  levels  delineated  in 
the  Training  Level  Assignments  (TLA)  for  that  TPC.   (The 
TPC,  TOS,  and  TLA  are  components  of  the  TPS.)   SATs  are 
administered  to  SSBN  personnel  during  their  off-crew  period. 
Second-level  maintenance  and  instructor  personnel  are 
tested  annually  with  SATs.   Each  SAT  version  remains  effective 
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for  administration  for  a  period  not  longer  than  6  months 
(for  SSBN  examinees)  or  12  months  (for  other  examinees), 
after  which  it  is  retired  and  replaced  with  one  different, 
but  constructed  to  the  same  design  specifications  (applicable 
portions  of  the  PPP  and  TPS) . 

CATs  are  administered  in  training  courses  to  measure 
training  effectiveness  (the  scope  of  which  includes  the 
quality  of  instruction,  training  facilities,  hardware,  and 
documentation  support)  and  the  level  of  trainee  comprehen- 
sion of  training  presented.   Each  CAT  is  applicable  to  a 
particular  course  or  major  portion  thereof,  and  consists  of 
knowledge  test  items  and/or  skill  test  devices  which  sample 
from  among  all  of  the  TOS  knowledge  depths  and  skill  levels 
delineated  for  that  course  or  course  portion  in  the  curricu- 
lum Profile  Item  to  Topic  Objective  Assignment  Chart  (OAC) 
for  that  course.   CAT  administration  occurs  immediately 
following  the  applicable  portion  of  training.   Each  CAT 
version  remains  effective  for  administration  for  a  period 
not  longer  than  12  months,  after  which  it  is  retired  and 
replaced  with  one  different,  but  constructed  to  the  same 
design  specifications  (applicable  portions  of  the  PPP  and 
TPS)  . 

Analysis  and  evaluation  is  the  component  of  PTEP  which 
provides  qualitative  assessment  of  the  Training  Program. 
It  is  the  process  through  which  personnel  testing,  data 
collection,  and  analysis  are  integrated  to  identify  defi- 
ciencies and  to  recommend  corrective  actions.   This  process 
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monitors  and  measures  the  effectiveness  of  the  Training 
Program,  and  thereby  serves  as  a  significant  basis  on  which 
improvements  are  determined  and  developed. 

Polaris/Poseidon  PTEP  analysis  and  evaluation  are  directed 
toward  four  major  areas:   personnel,  training,  PPP/TPS ,  and 
the  PTEP  personnel  tests.   These  analyses  and  evaluations 
are  performed  in  a  collective  manner  to  enable  the  identifi- 
cation of  trends  and  deficiencies  and  the  formulation  of 
corrective  action  recommendations  affecting  any  element  of 
the  Training  Program.   These  trends,  deficiencies,  and 
recommendations  are  reported  in  a  timely  manner  to  appro- 
priate Training  Program  management  activities  and  commands . 

Each  personnel  test  version  used  in  PTEP  is  evaluated 
to  determine  the  adequacy  and  efficiency  of  the  overall 
test,  as  well  as  its  constituent  test  instruments,  in  ful- 
filling the  test  design  specifications.   An  inherent  part  of 
this  evaluation  is  the  evaluation  of  the  test  design  speci- 
fications themselves,  to  determine  whether  they  adequately 
and  efficiently  serve  to  describe  the  test  vehicle  require- 
ments with  respect  to  the  overall  testing  objectives. 

The  adequacy  of  personnel  to  support  the  prescribed 
mission  is  evaluated  primarily  from  personnel  test  results. 
Evaluations  are  directed  toward  each  individual  participant, 
as  well  as  each  identifiable  group  of  participants  (e.g., 
all  technicians  of  a  common  NEC/TPC  and  of  a  common  SSBN 
crew) ,  and  consider  personnel  history  data  and  other 
pertinent  data,  as  applicable. 
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The  effectiveness  and  efficiency  of  training,  conducted 
as  part  of  the  Training  Program,  are  evaluated  from  the 
training  materials,  the  criteria  on  which  the  training  is 
based,  and  personnel  test  results.   Evaluations  are  conducted 
to  determine  whether  the  training  fulfills  the  requirements 
set  forth  in  the  PPP  and  TPS,  and  whether  duplicate  training 
exists  among  related  courses  or  course  segments. 

The  accuracy  and  currency  of  the  PPP  and  TPS  are  evalu- 
ated with  respect  to  the  operational  hardware  and  software. 
Evaluations  are  also  conducted  to  determine  the  effective- 
ness of  the  PPP  and  TPS  in  serving  as  definitive  standards 
for  all  other  elements  of  the  Training  Program. 

Several  report  types  are  used  to  disseminate  PTEP 
personnel  test  results  and  evaluation  information  (i.e., 
trends,  deficiencies,  conclusions,  and  recommendations). 
These  reports  provide  for  the  following: 

1.  Immediate  feedback  of  PTEP  personnel  test  results. 

2.  Reporting  of  follow-on  results  of  detailed  analysis 
and  evaluation  performed  after  each  PTEP  test 
version  is  retired. 

3.  Immediate  feedback  of  identified  Training  Program 
deficiencies  and  recommended  corrective  actions. 

4.  As-required  progress  reporting  of  personnel  indica- 
tions, including  current  performance  levels,  con- 
clusions, and  related  training  and  documentation 
data. 

The  amount  of  data  routinely  processed  within  PTEP  is 

such  that  EDP  support  is  required  to  provide  the  necessary 

timeliness  and  efficiency.   EDP  support  is  used  for  direct 
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support  of  PTEP  data  collection,  analysis,  and  reporting 
activities.   Polaris/Poseidon  PTEP  used  EDP  to  facilitate 
test  generation,  test  scoring  and  reporting,  personnel  test 
and  nontest  data  collection,  and  analysis  of  personnel, 
curricula,  and  training  facilities  data. 

The  PTEP  EDP  system  is  composed  of  five  major  subsystems. 

1.  The  Test  Generation  Subsystem  includes  programs  for 
maintenance  of  the  test  item  and  test  reference  files,  and 
programs  for  generation  and  maintenance  of  knowledge  test 
parts  for  SATs  and  CATs ,  scoring  keys  for  knowledge  and 
skill  test  parts,  and  other  data  used  in  scoring  and  reporting 
functions.   Subsystem  requirements  are  detailed  in  DDL 
Specifications  TEG  100,  TEG  110,  and  TEG  120. 

2.  The  Test  Scoring  and  Reporting  Subsystem  includes 
programs  to  accept,  edit,  and  store  raw  test  data  (examinee 
answer  sheets  and  skill  test  scoring  sheets),  and  to  assemble, 
score,  and  report  test  results.   Teleprocessing  programs 

are  also  provided  to  facilitate  remote  input/output  capa- 
bilities.  Subsystem  requirements  are  detailed  in  DDL 
Specification  TEG  200. 

3.  The  Personnel  Subsystem  stores  and  maintains  records 
for  Poseidon  enlisted  personnel,  and  selects  and  prints 
formal  Personnel  Data  Sheets  for  evaluation  and  administra- 
tive purposes.   Subsystem  requirements  are  detailed  in  DDL 
Specifications  TEG  300  and  TEG  310. 

4.  The  Test  Analysis  Subsystem  analyzes  test  data 
collected  during  the  effective  "life"  of  a  given  personnel 
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test  version.   Programs  are  included  which  analyze  scores 
to  support  personnel,  curricula,  and  facility  evaluation, 
and  which  compute  and  report  a  variety  of  statistical  data 
to  support  maintenance  and  improvement  of  the  PTEP  test 
instruments.   Basic  subsystem  requirements  are  documented 
DDL  Specification  TEG  400.   An  additional  program  computes 
inter-score  correlations  and  displays  frequency  distributions 
of  scores. 

5.   The  Query  Subsystem  provides  the  capability  to 
support  special  studies  and  investigations  by  retrieving 
pertinent  information  from  the  EDP  files.   Preprocessor 
programs  accept  user-defined  record  selection  data  reporting 
instructions  and  prepare  an  EDP  program  to  execute  those 
instructions.   Subsystem  requirements  are  detailed  in  DDL 
Specification  TEG  500. 

The  Poseidon  PTEP  EDP  system  is  installed  at  the  Data 
Processing  Facility,  Polaris  Missile  Facility-Atlantic 
(POMFLANT) ,  Charleston,  South  Carolina.   The  following  items 
are  the  significant  features  and  characteristics  of  that 
system. 

1.  Computer.   IBM  System  360,  Model  F30,  is  used,  with 
core  extended  to  a  capacity  of  96K  bytes. 

2.  Operating  System.   Disk  Operation  System  (DOS)  is 
used. 

3.  Mass  Memory.   Mass  memory  consists  of  Model  2314  disk 
storage  facilities.   (At  most,  five  disk  drivers  are  used  at 
any  time. ) 
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4.  Input.   Data  inputs  are  accomplished  by  Model  2540 
card  reader  and  Model  2701  data  adapter  unit  with  appro- 
priate data  sets  (for  teleprocessing  applications) . 

5.  Output.   Data  outputs  are  accomplished  by  Model  1403 
line  printer,  Model  2540  card  punch,  and  the  same  tele- 
processing interface  devices  used  for  data  input. 

6.  Data  Storage.   Data  storage  devices  are  removable 
disk  packs  for  use  in  the  2314  disk  facility.   Data  are 
stored  on  six  "current"  and  four  "backup"  disk  packs. 

7.  Computer  Software  and  Utility  Programs.   The 
Poseidon  PTEP  EDP  system  uses  ANS  COBOL  and  FORTRAN  IV 
compilers,  plus  utility  programs  for  card-to-disk  copy  and 
disk-sort.   Basic  assembly  language  (BAL)  programs  using 
basic  teleprocessing  access  method  (BTAM)  instructions  con- 
trol teleprocessing  functions. 

8.  Remote  Access.   Poseidon  PTEP  test  sites  at  Guided 
Missiles  School,  Dam  Neck,  Virginia;  Submarine  Base,  New 
London,  Connecticut;  and  FBM  Training  Center,  Charleston, 
South  Carolina  use  AUTOVON  phone  lines  to  input  personnel 
test  data  and  receive  test  reports.   Test  site  facilities 
include : 

a.  Optical  Scanning  Device.   OPSCAN  Corporation 
Model  17  scanner  is  used  to  "read"  raw  test  data  and  test 
scoring  requests  from  paper  into  machine  compatible  forms. 

b.  Teletypewriter.   Western  Union  Model  ASR  33 
teletype  with  appropriate  data  set  is  used. 
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Documentation  is  prepared  and  maintained  current  to 
describe  PTEP  and  to  set  forth  its  detailed  implementa- 
tion procedures  and  data  forms.   Procedures  and  forms  are 
documented  for  the  personnel  testing,  analysis  and  evaluation 
(including  nontest  data  collection) ,  and  EDP  components  of 
PTEP.   Polaris/Poseidon  PTEP  description  and  procedures  are 
documented  in  NAVORD  OD  45953.   This  documentation  is  main- 
tained current  and  effective  through  continuous  monitoring 
of  the  personnel  testing,  analysis  and  evaluation,  and  EDP 
components  of  PTEP.   Changes  to  NAVORD  OD  45953  are  prepared 
and  issued  as  required  to  reflect  the  actual  implementation 
procedures  and  data  forms  employed,  as  they  are  modified 
to  improve  the  effectiveness  of  PTEP. 

D.   VTS  BACKGROUND 

The  Versatile  Training  System  (VTS) ,  a  development  of 
Naval  Weapons  Center  California,  is  presently  planned  to 
provide  Test  and  Information  Handling  (IH)  support  for 
TRIDENT-1  PTEP  functions.   The  VTS  is  designed  to  provide 
all  training  support  required  to  improve  the  effectiveness 
of  training  both  officer  and  enlisted  personnel  of  Naval 
Aviation  Fleet  Readiness  Squadrons,  Naval  Aviation  Opera- 
tional Squadrons,  Naval  Aviation  Maintenance  Training 
Detachments,  U.S.  Marine  Corps  Aviation  Training  Activities, 
Naval  Air  Station  and  Marine  Corps  Air  Station  Aircraft 
Intermediate  Maintenance  Departments,  TRIDENT  SSBNs ,  and 
Submarine  Training  Support  Activities.   The  VTS  looked 
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promising  from  several  viewpoints.   It  was  driven  by  a 
popular,  extremely  powerful  commercial  minicomputer  of 
relatively  small  cost  (compared  to  large  scale  systems) . 
Additionally,  it  could  be  purchased  as  a  Federal  Procurement 
Schedule,  Group  66  item,  thereby  facilitating  the  procedures 
for  its  procurement.   Realizing  the  apparent  efficiency  of 
the  multi-application  of  VTS ,  PM-2  placed  an  order  with 
NWC,  China  Lake  to  prepare  and  install  a  VTS  at  TRITRAFAC 
by  August,  1978  [Ref .  26] .   It  is  presently  planned  that 
each  TRIDENT  SSBN  off-crew  office,  the  FBM  Training  Center 
and  the  Submarine  Group  would  have  a  remote  terminal  to 
access  real  time  the  personnel  training  data  files.   Also 
presently  planned  under  TRIDENT  is  the  placement  of  a  VTS 
remote  terminal  at  PERS  5C,  the  Enlisted  Submarine  Detailer 
in  Washington,  D.C.   NNC ' s  tasks  were  to  include  responsi- 
bilities for  developing  TRIDENT-unique  software  and  for 
programming  and  integrating  the  PTEP  software  into  the 
system. 

In  November  of  1976  the  Officer  in  Charge,  Central  Test 
Site  for  PTEP  recommended  to  SSPO  that  action  be  initiated 
to  procure  a  VTS  to  support  Polaris/Poseidon  and  TRIDENT-1 
Backfit  PTEP  programs  [Ref.  27].  Approval  was  granted  for 
VTS  implementation  to  support  Poseidon  and  TRIDENT-1  Backfit 
PTEP  programs  with  scheduled  installation  and  operation  to 
coincide  with  TRIDENT  PTEP  completion  in  FY  78. 
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E.   PTEP  MODIFICATION  WITH  VTS 

The  VTS  as  presently  planned  will  provide  PTEP  with 
their  existing  system  with  the  addition  of  an  "on-line" 
mode  with  the  capability  of  many  users  (63  with  future  expan- 
sion to  128)  interacting  with  the  computer  equipment  simul- 
taneously on  different  jobs.   The  presently  planned  PTEP 
option  includes  a  PDP  11-70  mainframe  at  Guided  Missiles 
School  (GMS) ,  Dam  Neck,  plus  incremental  Peripheral  equipment 
increase  to  accommodate  removal  of  PTEP  data  base  from 
POMFLANT  computer  and  storage  onto  the  PDP  11-70  at  GMS, 
Dam  Neck.   This  option  would  also  provide  Charleston,  S.C. 
and  New  London,  Connecticut  CTS  detachments  with  a  PDP  11-60 
and  peripherals.   The  present  data  retrieval  system  at 
POMFLANT  is  slow  and  cumbersome  [Ref .  27] .   The  ability  of 
CTS  to  answer  management  questions  in  a  realistic  time  frame 
is  severely  limited  by  EDP  support.   For  example,  a  simple 

question  as  "How  many  NEC  personnel  are  not  conversion 

trained?"  would  take  a  minimum  of  two  days  and  more  commonly 
a  week  to  answer.   With  VTS  the  answer  could  be  obtained 
in  approximately  thirty  minutes. 

Other  benefits  that  will  come  with  PTEP  as  a  part  of 
VTS  will  be  commonality  with  TRIDENT  information  handling, 
shared  cost  in  updating,  reduced  requirement  for  contractor 
support  personnel,  more  efficient  use  of  Naval  Personnel, 
improved  measurement  capabilities,  cost  significantly  less 
than  the  present  system  to  operate,  and  have  a  greater 


potential  for  growth  [Ref .  27] .   A  typical  submarine  VTS 
is  depicted  in  Figure  V-2.   Appendix  B  contains  excerpts 
from  Digital  Equipment  Corporation's  Resource  Time  Sharing 
System/Extended  (RTSE/E)  brochure  explaining  the  data 
management  system  used  [Ref.  28]. 

F.   ALTERNATIVES 

1.   Modify  the  Present  Management  Information  System 

The  existing  PTEP,  which  is  currently  being  imple- 
mented for  Poseidon/TRIDENT-1  Backfit  with  VTS  could  be 
used  as  a  baseline  for  development  of  a  SOTAP  IFAS.   The 
VTS  can  be  expanded  to  accept,  store,  and  manipulate  data 
from  other  interactive  training  measurement  devices  [Ref.  27], 
This  gives  PTEP  measurement  output  capabilities  in  areas 
where  none  currently  exist.   A  Sonar  personnel  testing 
baseline  (SAT's  and  CAT's)  is  currently  under  development 
by  the  PTEP  CTS. 

The  measurement  of  sonar/fire  control  team  personnel 
proficiency  would  be  accomplished  through  the  administration 
of  standardized  training  and  assessment  exercises  which 
would  be  based  on  the  sonar/fire  control  team  knowledge  and 
skill  requirements  set  forth  in  the  Sonar  Team  Performance 
Profile  (STPP)  and  the  Fire  Control  Team  Performance 
Profile  (FCTPP) ,  both  of  which  would  be  added  elements  of 
the  Training  Program  to  be  developed. 

Another  necessary  modification  would  be  the  optically 
scanned  data  scoring  sheet.   Currently,  as  shown  in  Appendix  C, 
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the  data  sheet  is  designed  for  scoring  one  person's  data 
per  sheet.   Modifications,  such  as  condensing  the  fields 
as  has  been  done  partially  on  the  top  of  the  present  form, 
would  make  it  possible  to  enter  many  trainee's  results  on 
one  page  reducing  the  amount  of  paper  handling  necessary 
for  inputing  to  the  computer. 

Equally  important  and  most  likely  the  most  critical 
is  the  modification  and/or  development  of  the  software 
packages.   As  the  complexity  of  software  has  grown  there 
has  been  an  ever-increasing  time  lag  in  meeting  these  needs 
and  maintaining  the  programs .   The  problems  and  resulting 
expense  involved  has  become  all  but  prohibitive.   Seemingly, 
as  one  set  of  needs  are  met  others  present  themselves  and 
usually  the  entire  programs  have  to  be  redesigned  [Ref .  8] . 

Another  disadvantage  is  that  the  present  IH  system 
may  not  be  completely  modifiable  to  obtain  the  results 
desired  for  the  SOTAP  IFAS. 

One  of  the  more  propitious  aspects  of  this  alterna- 
tive is  that  the  information  system  will  not  have  to  be  pur- 
chased; therefore,  no  large  capital  investment  is  required. 
SOTAP  would  just  have  to  pay  for  the  modification  of  the 
present  system  to  accommodate  SOTAP  needs.   Another  distinct 
advantage  is  the  use  of  existing  Naval  personnel  at  the 
PTEP  CTS  and  detachments.   Once  a  part  of  PTEP ,  the  cost 
of  updating  PTEP  data  handling  functions  could  therefore 
be  shared  amongst  the  several  users.   The  commonality  with 
the  TRIDENT  Information  Handling  (IH)  would  permit  several 
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realizable  benefits  such  as  easy  exchange  of  data  and  the 
efficiencies  recognized  by  CTS/DIRSSP  managing  a  single 
IH  system;  TRIDENT,  TRIDENT-1  Backf it ,  Poseidon  common  data 
would  not  require  duplicate  handling  and  storage;  and, 
improved  capability  in  testing  or  data  presentation  for 
either  TRIDENT,  POSEIDON,  or  the  NAVAIR  system  would  be 
realized  in  the  SOTAP  IFAS  at  no  additional  cost. 

2 .   Develop  a  New  Management  Information  System 

The  results  of  developing  a  new  management  infor- 
mation system  for  the  SOTAP  IFAS  which  may  or  may  not  be 
compatible  with  other  submarine  training  information  handling 
systems  has  one  distinct  advantage;  in  that,  from  ground- up 
the  system  can  be  designed  and  tailored  to  the  specific 
needs  of  the  SOTAP  IFAS.   If  the  decision  is  made  to  acquire 
a  new  system,  NUSC  must  then  approach  the  problems  which 
will  accompany  such  an  endeavor.   These  problems  will,  of 
course,  vary  to  some  degree  with  the  acquiring  activity  and 
the  equipment  system  to  be  acquired.   However,  considerations 
involved  in  the  selection  of  equipment,  the  acquisition  and 
training  of  qualified  personnel,  the  plans  necessary  in 
acquiring  the  new  system,  the  provision  for  the  physical 
facilities  needed  by  the  computer  and  its  associated  peripheral 
equipment  and  the  cost  of  installations  and  operations  are 
just  some  of  the  common  features  brought  out  in  Chapter  IV 
regardless  of  the  particular  system  to  be  installed. 

In  the  case  of  the  Navy  it  is  the  office  of  the 
Automatic  Data  Processing  Equipment  Selection  Office  (ADPESO) 
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established  in  July,  1967  that  is  charged  with  the  overall 
coordination  of  automatic  data  processing  equipment  (ADPE) 
requirements.   Prior  to  its  formation  the  selection  of 
ADPE  was  accomplished  by  the  various  heads  of  departmental 
components.   A  full  time  staff  was  hired  and  the  responsi- 
bility for  selection  was  centralized  and  elevated  to  a 
higher  level  in  the  Department  of  the  Navy,  a  field  activity 
under  command  of  the  Chief  of  Naval  Operations. 

With  the  acquisition  of  any  complex  system,  schedule 
and  available  funding  are  key  issues  to  consider.   Funding 
available,  the  acquisition  process  is  still  a  lengthy  process 
under  ADPESO  which  has  five,  very  rigid  and  extremely  time 
consuming,  steps  in  their  computer  procurement  process. 
If  funding  is  not  available  either  for  purchase  or  lease 
then  investigation  into  the  possibility  of  sharing  equipment 
with  other  government  agencies  in  the  local  area  or  to 
acquire  unused  government-owned  equipment  through  the 
reutilization  program.   The  General  Services  Administration 
publishes  a  periodic  summary  of  all  government-owned  equip- 
ment not  presently  being  used  that  can  be  acquired  for  only 
the  cost  of  packing  and  transportation.   Pertinent  directives 
are  DOD  INST  4160. 19M  and  SECNAVINST  10462.17. 
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VI.   CONCLUSIONS  AND  RECOMMENDATIONS 

A.   CONCLUSIONS 

What  seems  to  be  clear  to  the  author,  as  the  Navy  moves 
into  the  1980 's,  is  that  more  is  being  required  than  can  be 
accomplished.   The  inevitable  result  is  low  quality  perform- 
ance, unfulfilled  requirements,  or  both  —  and  both  are 
unacceptable.   The  situation  is  being  aggravated  further  by 
continuing  reductions  in  Navy  force  levels  and  other  economy 
measures  invoked  without  equally  compensating  reductions  in 
missions  and  requirements.   For  example,  if  the  Defense 
Department  budget  this  year  goes  through  as  proposed,  the 
Navy  is  going  to  lose  about  11,700  authorized  billets,  most 
of  them  to  come  out  of  the  training  "pipeline"  [Ref.  29]. 

What  is  the  answer  to  this  dilemma?   Obviously,  we  in 
the  Navy  cannot  control  national  commitments.   We  cannot 
effect  the  technological  gains  of  our  potential  enemies, 
nor  would  we  wish  to  slow  down  the  pace  of  our  own  techni- 
cal growth.   Yet,  all  these  things  contribute  to  escalating 
commitments  and  requirements.   The  author  believes  that  the 
Sonar  Operational  Training  and  Assessment  Program  is  an 
outstanding  imaginative  idea  to  deal  with  these  problems 
and  to  harness  our  technology  to  serve  us  in  a  way  that 
reduces,  not  increases,  individual  training  effort. 
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B.   RECOMMENDATIONS 

Making  decisions  would  be  relatively  easy  if  all  one 

had  to  do  was  look  at  the  analysis  of  all  the  alternatives 

and  choose  the  most  beneficial.   However,  James  Schlesinger 

writing  to  the  Senate  Committee  on  Government  Operations  in 

1968  stated  that  analysis  has  been  greatly  oversold  [Ref.  30] 

In  recent  years  it  has  been  recognized  in 
public  statements  (as  well  as  the  textbooks) 
that  analysis  is  not  a  scientific  procedure 
for  reaching  decisions  which  avoid  intuitive 
elements,  but  rather  a  mechanism  for  sharpen- 
ing the  intuitions  of  the  decisionmaker  ...  No 
matter  how  large  a  contribution  that  analysis 
makes,  the  role  of  subjective  preference  of  the 
decisionmaker  remains  imposing.   Analysis  is, 
in  the  end,  a  method  of  investigating  rather  than 
solving  problems.   [Ref.  30]. 

There  is  a  difference  between  the  quantifiable  and 
unquantif iable.   The  decisionmaker  must  look  at  and  evalu- 
ate more  than  just  the  quantifiable  aspects  of  the  alterna- 
tives.  Using  experience  and  judgment,  one  must  attempt  to 
put  subjective  values  on  unquantif iables .   However,  there 
is  not  enough  information  about  uncertainties  to  absolutely 
quantify  the  unquantif iables;  therefore,  the  author's 
recommendations  must  be  more  biased  toward  using  previous 
experience  and  judgment  based  on  investigative  efforts 
undertaken  during  this  thesis  endeavor. 

After  weighing  all  the  advantages  and  disadvantages 
of  the  alternatives ,  the  author  would  recommend  Alternative 
1  —  Modify  the  present  management  information  system.   Since 
there  are  many  uncertainties  in  choosing  any  alternatives, 
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the  author  feels  that  selecting  Alternative  1  will  allow 
SOTAP  to  keep  the  most  options  open  at  the  least  cost. 
The  author  believes  that  the  currently  budgeted  dollars  in 
the  SOTAP  program  should  be  adequate  for  PTEP  modification. 
Probably  the  most  important  reason  has  to  do  with  "guaranteed 
satisfaction. "   It  would  be  a  terrible  mistake  to  make  a 
large  capital  investment  and  be  dissatisfied.   Management 
would  be  upset  for  making  the  wrong  decision  in  addition 
to  paying  more  for  that  choice. 

Further  recommendations  include  installing  a  VTS  remote 
terminal  at  SUBLANT,  Norfolk,  Virginia,  to  provide  the 
Type  Commander  access  to  sonar  operational  performance 
evaluations.   This  action  would  also  allow  the  Type  Commander 
access  to  any  Submarine  VTS  information.   The  major  advantage 
being  the  reduction  of  paper  report  submissions.   Providing 
VTS  remote  terminals  at  NUSC,  each  SSBN  off -crew  office, 
each  FBM  Training  Center,  and  each  Submarine  Group  in  New 
London,  Connecticut,  and  Charleston,  South  Carolina,  is 
recommended.   This  would  allow  real  time  access  to  the  data 
files  and  complete  the  information  flow  chain.   Facilities 
are  available  in  Digital  Equipment  Corporation's  RSTS/E 
system  for  sending  messages  to  all  terminal  users,  thus 
providing  a  useful  means  of  information  flow  between  shore- 
based  training  sites.   In  addition,  quarterly  NUSC  sponsored 
sonar  operational  training  meetings  for  FBM  Training  Centers 
sonar  personnel  including  SOT  instructors  and  off-crew 
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status  SSBN  sonar  personnel  in  New  London,  Connecticut, 
and  Charleston,  South  Carolina,  to  facilitate  a  free  flow 
of  sonar  operational  training  information  flow  is  recommended. 
Properly  scheduled  quarterly  meetings  would  ensure  that  all 
SSBN  sonar  teams  (users)  would  be  involved  in  the  training 
information  feedback. 

The  author  also  recommends  for  user  feedback  to  use  the 
SSBN  Weapon  System  Trouble  and  Failure  Report/Training 
Material  Change  Recommendation  (TFR/TMCR)  system  for 
recommending  changes  to  SOTAP  materials.   The  mechanism 
is  already  in  existence  and  FBM  Weapon  System  personnel 
including  Sonar  Technicians  are  familiar  with  the  system. 
TFR's  are  presently  required  on  Training  problems.   NAVSEA 
OD  28385  Volume  I  (TFR  Instructions)  discusses  training 
problems  as  related  to  Training  Management  Documentation 
(i.e.,  OD  45953-PTEP  Manual).   With  slight  modifications  to 
NAVSEA  OD  28385  Volume  I,  other  applicable  publications, 
directives,  and  instructions  a  user-feedback  information 
flow  reporting  system  could  be  implemented  for  the  SOTAP 
I  FAS. 

C.   AUTHOR'S  COMMENTS 

Advanced  education,  coupled  with  personal  experience, 
enables  one  to  develop  the  necessary  management  acumen  to 
effectively  cope  with  the  future.   Management  courses  such 
as  those  offered  at  the  Naval  Postgraduate  School  provide 
managers  insights  into  management,  organizational  behavior, 
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and  systems  which  increase  their  capability  to  be  effective 
managers.   However,  one  can  sit  in  the  classroom  gathering 
knowledge  about  the  principles  of  management  until  eternity 
and  still  not  become  an  effective  manager.   One  must  get 
into  the  environment  and  understand  the  climate  before  he 
can  begin  to  manage  effectively.   For  this  reason  the  author 
wanted  to  examine  the  real  environment  of  project  management 
and  learn  first  hand  how  things  are  done  (i.e.  uniting  theory 
and  practice) ,  rather  than  write  a  thesis  only  from  library 
research. 

Working  with  the  SOTAP  Program  Management  at  NUSC,  New 
London,  and  applying  systems  acquisition  management  princi- 
ples acquired  at  the  Naval  Postgraduate  School  has  been  a 
gratifying  experience.   Bridging  the  gap  between  education 
and  the  real  environment  has  cemented  the  foundation  of 
knowledge. 
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APPENDIX  A 


SYSTEMS  OVERVIEW  DRAWING  OF 
AN  INFORMATION  SYSTEM  DEVELOPMENT  PROCESS 
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APPENDIX  B 
RESOURCE  SHARING  TIMESHARING  SYSTEM/EXTENDED  (RSTS/E)  SUMMARY 

A.   GENERAL 

RSTS/E  (Resource  Sharing  Timesharing  System/Extended)  is 
the  primary  timesharing  system  for  the  PDP-11  Family.   It 
provides  general  timesharing  facilities  through  the  BASIC- 
PLUS  language,  an  enriched  version  of  Dartmouth  Standard 
BASIC.   An  optional  batch  COBOL  facility  is  available  to 
enhance  the  business  data  processing  requirements  of  certain 
applications.   The  system  features  complete  system  utiliza- 
tion from  an  interactive  terminal,  with  a  large  number  of 
such  terminals  being  active  concurrently,  through  flexible 
combinations  of  local,  remote,  and  multiplexed  interfaces. 

The  RSTS/E  system  requires  a  PDP-11  systems-level 
computer  (PDP-11/35,  11/40,  11/45),  32K  words  of  parity 
memory,  hardware  memory  management,  and  disk  storage  with 
adequate  backup.   User  access  and  file  protection  are  pro- 
vided, and  RSTS/E  supports  a  wide  range  of  PDP-11  peripherals. 
For  a  normal  mix  of  jobs,  up  to  32  concurrent  users  can 
be  supported  on  a  PDP-11/45  system,  and  up  to  24  concurrent 
users  on  a  PDP-11/35  or  11/40. 

To  make  full  use  of  the  power  of  the  PDP-11/70,  RSTS/E 
has  been  expanded  to  accommodate  up  to  63  concurrent  users. 
The  system  supports  the  high-performance  peripherals  necessary 
to  ensure  the  continuous  performance  for  the  large  numbers 
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of  users,  and  the  flexibility  to  provide  interactive  data 
base  management  for  business  applications,  as  well  as  the 
scientific  resources  for  the  general  timeshared  applications 
commonly  found  in  educational  environments . 

B.   RESOURCE  SHARING 

RSTS/E  users  have  on-line  access  to  a  wide  range  of 
program  and  data  files.   Files  may  be  created,  updated, 
extended  and  deleted  from  the  user's  terminal  or  under 
program  control.   Up  to  12  files  may  be  open  at  any  one 
time.   Since  files  may  be  opened  and  closed  during  the 
running  of  a  program,  the  actual  number  referenced  in  a 
program  may  be  far  greater  than  12 .   The  total  number  of 
files  a  user  may  have  stored  in  a  disk  library  is  bounded 
only  by  the  total  system  disk  capacity  and  the  library 
demands  of  the  other  users. 

RSTS/E  files  are  not  limited  to  a  disk  files.   Data 
may  be  read-in  from  a  card  reader  and  printed  on  a  high- 
speed printer.   The  on-line  user  can  assign  devices,  even 
other  terminals,  for  input  and  output  functions  through  his 
programs.   Thus,  individual  users  get  exclusive  use  of 
these  devices  for  as  long  as  required,  then  release  them 
for  others  to  use.   This  is  known  as  "resource  sharing." 

Private  data  files  may  be  stored  on  removable  disk 
cartridges,  disk  packs,  DECtape  or  paper  tape.   Confidential 
files  may  be  dismounted  when  not  in  use  and  kept  under  lock 
and  key.   These  stored  files  may  be  as  large  as  33.5  million 
bytes,  yet  accessible  on  a  completely  random  basis. 
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C.  THREE  TYPES  OF  DATA 

RSTS/E  has  the  capability  to  handle  three  types  of 
data-floating  point,  integer  and  character  string.   Floating- 
point numbers  are  used  for  most  numeric  representation  and 
may  be  one  of  two  levels  of  precision:   7  decimal  digits 

(two  computer  words)  or  17  digits  (four  words) .   Number 

3  8       -3  8 
size  may  vary  from  approximately  10    to  10 

Integers  may  be  used  for  greater  processing  efficiency 
as  indices,  counters  and  subscripts.   They  are  whole  numbers 
in  the  range  -32,768  to  32,767. 

Character  strings  are  available  for  powerful  processing 
of  non-numeric  data.   Strings  may  be  as  short  as  a  single 
character  or  unlimited  in  length.   Strings  used  in  virtual 
memory  are  limited  to  512  characters.   Groups  of  strings, 
a  list  of  names  and  addresses  for  example,  may  be  organized 
in  tables  or  arrays  just  like  numeric  information.   Since 
strings  can  be  read  from  or  written  to  external  files  in  a 
sequential  or  random  manner,  whole  files  of  textual  data  may 
be  built  up  and  updated  on-line. 

D.  VIRTUAL  ARRAYS 

The  concept  of  virtual  memory  essentially  makes  the 
system  disks  an  extension  of  main  memory.   This  permits  the 
user  to  manipulate  large  arrays  of  tables  of  data  without 
cutting  into  program  size  and  indeed,  process  larger  masses 
of  data  than  will  fit  in  the  entire  main  memory  of  the 
system.   Furthermore,  the  user  can  access  large  amounts  of 
data  without  the  need  for  explicit  read/write  programming. 
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Data  in  virtual  memory  arrays  may  also  be  processed 
using  MATRIX  statements.   These  statements  perform  opera- 
tions on  multiple  elements  of  virtual  memory  arrays  with 
a  single  statement. 

Virtual  memory  may  be  used  to  store  any  type  of  data  — 
floating-point,  integer  or  character  string.   Floating- 
point virtual  memory  might  be  used  by  an  industrial  dis- 
tributor to  store  customer  account  balances  on  a  daily 
basis.   Character  string  virtual  memory  could  be  used  to 
store  names  and  course  preferences  for  a  college  on-line 
registrations  system. 

RSTS/E  uses  a  system  of  in-core  256-word  buffers  when 
processing  virtual  memory  arrays.   With  this  system,  a 
disk  transfer  is  not  necessarily  made  every  time  a  virtual 
memory  variable  is  referenced.   Consequently,  virtual  memory 
is  as  mindful  of  processor  efficiency  as  it  is  of  programming 
ease. 

E.   MULTIPLE-USER  ACCESS  TO  COMMON  FILES 

It  is  often  desirable  to  have  one  or  more  on-line  disk 
files  simultaneously  accessible  to  more  than  one  user.   For 
example,  in  an  order  entry/inventory  control  system,  several 
clerks  might  be  entering  orders  and  each  must  have  access 
to  the  same  customer  master  file  and  inventory  control  file. 
Or  in  a  college  on-line  registrations  system,  students  would 
register  and  enter  their  course  preferences  at  a  number  of 
terminals  simultaneously. 
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Under  RSTE/E,  any  number  of  users  may  read  data  from 
the  same  file  simultaneously.   Typically,  only  one  user  at 
a  time  may  write  on  the  file.   However,  when  multiple-user 
updating  is  desirable,  as  previously  described,  the  UPDATE 
feature  permits  this  to  be  handled  safely  by  locking  out  a 
physical  disk  record  from  other  users  while  one  user  is  in 
the  process  of  updating  the  record.   While  the  record  is 
locked  out,  other  users  are  temporarily  prevented  from 
accessing  it,  although  they  can  read  or  write  any  other 
record  in  the  file  not  currently  locked  out.   When  the 
locked-out  record  is  updated,  it  then  once  again  becomes 
accessible  to  other  users.   In  this  way,  all  users  are 
guaranteed  access  only  to  current,  valid  records  instead  of 
records  that  are  not  up  to  date  because  they  are  in  the 
process  of  being  altered  by  another  user. 

F.   BASIC-PLUS,  AN  EXPANDED  LANGUAGE 

Timesharing  users  interact  with  RSTS/E  using  BASIC-PLUS. 
The  language  is  easy  to  learn  and  work  with,  yet  puts  the 
enormous  power  of  the  system  at  the  user's  fingertips.   The 
immediate  mode  of  operation  enables  the  terminal  to  be  used 
for  simple  calculations.   Dynamic  debugging  is  faster  since 
programs  may  be  interrupted  at  any  point,  checked,  corrected, 
and  operation  resumed. 

BASIC-PLUS  automatically  checks  all  program  commands 
for  accuracy  when  they  are  entered.  Errors  are  reported 
immediately.   Since  each  program  line  is  compiled  as  it  is 
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entered,  there  are  no  frustrating  delays,  even  on  the  RUN 
command. 

BASIC-PLUS  is  a  significant  extension  of  Dartmouth 
BASIC  to  increase  its  utility  and  make  RSTS/E  the  ideal 
tool  to  solve  a  broad  range  of  problems.   For  example, 
administrative  applications  such  as  on-line  order  entry, 
inventory  control  and  payroll  may  be  implemented  efficiently 
by  using  language  features  suited  for  data  processing.   Text- 
processing  applications  such  as  Computer  Assisted  instruction, 
(CAI) ,  automated  letter  or  document  editing  and  production 
may  utilize  the  set  of  character  string  handling  functions. 
The  utility  of  BASIC  for  computational  applications  such 
as  structural  design  and  simulation  is  extended  with  language 
features  which  allow  more  concise,  and  therefore,  more 
efficient  programming  and  program  execution.   BASIC-PLUS 
eliminates  the  constraints  of  BASIC  for  a  variety  of 
applications  programming  tasks. 

Calculations  in  BASIC-PLUS  are  generally  executed  using 

floating-point  variables.   The  magnitude  range  of  numbers 

—38  +3  8 

lies  between  0.14  x  10     and  1.7  x  10    .   Two  levels  of 

precision  are  available:   7  decimal  digits  (two  computer 

words)  or  17  decimal  digits  (four  computer  words) .   The 

degree  of  precision  used  is  a  system  generation  parameter. 

Whichever  is  chosen  applies  to  all  users  of  the  system 

unless  the  system  is  regenerated  for  a  different  degree 

of  precision. 
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BASIC-PLUS  also  allows  the  use  of  integers.   These  are 
whole  numbers  in  the  range  -32,768  to  32,767.   The  most 
common  uses  of  integers  are  in  counting,  indexing  and  sub- 
script operations.   Since  integers  only  occupy  one  computer 
word,  their  use  often  increases  the  execution  efficiency 
of  programs. 

BASIC-PLUS  provides  a  comprehensive  set  of  mathematical 
functions  to  the  user  —  trigonometric,  logarithmic,  absolute 
value,  truncation,  pi,  random  number  generator  and  square 
root.   Logical  and  relational  operators  are  also  available. 

G.   IMMEDIATE  MODE  OF  EXECUTION 

Normal  timesharing  use  of  RSTS/E  consists  of  typing 
program  text  using  a  keyboard  terminal  and  at  the  end  of 
the  program  typing  a  RUN  command  at  which  time  the  program 
executes.   A  second  mode  of  using  RSTS/E,  called  immediate 
mode,  consists  of  typing  program  statements  on  the  keyboard 
and  having  them  executed  immediately.   Program  statements 
are  identified  in  either  case  except  that,  in  immediate  mode, 
they  are  typed  without  line  numbers. 

Two  uses  of  immediate  mode  might  be  1)  performance  of 
simple  calculations  in  situations  which  do  not  occur  with 
sufficient  frequency  to  justify  writing  a  program  and 
2)  program  debugging.   To  debug  a  program  a  user  can  place 
the  STOP  statement  liberally  throughout  the  program.   Each 
STOP  statement  causes  the  program  to  halt  and  prints  the 
line  number  at  which  the  STOP  occurred,  at  which  time  the 
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user  can  examine  and  change  various  data  values  in  immediate 
mode  and  give  a  command  to  continue  program  execution. 

H.   MATRIX  OPERATIONS 

The  user  of  RSTS/E  may  improve  processing  and  programming 
efficiency  by  organizing  his  numeric  data  into  one-  and  two- 
dimensional  arrays  or  matrices.   The  BASIC-PLUS  matrix 
commands   add,  subtract,  multiply  and  invert  entire  data 
matrices  in  a  single  operation.   Commands  are  also  available 
to  initialize  a  matrix  to  zeroes,  ones,  or  to  the  identity 
matrix. 

Both  numeric  and  character  string  matrices  may  be  input, 
read,  and  printed  with  single  commands.   If  the  matrices 
won't  fit  in  main  memory  the  BASIC-PLUS  virtual  memory  facility 
can  be  used  as  an  extension  of  main  memory  as  needed.   Thus, 
array  size  never  restricts  program  size,  or  vice  versa; 
RSTS/E  offers  unlimited  array  capability  even  with  the 
largest  programs. 

I.   EXTENDED  PROGRAM  STATEMENT  CODING 

The  effectiveness  of  RSTS/E  in  solving  problems  in  a 
broad  variety  of  application  areas  is  significantly  increased 
with  the  addition  of  numerous  extensions  to  the  structure 
(syntax)  of  the  BASIC  program  statements.   These  highly 
flexible  program  statements,  previously  found  only  in  advanced 
scientific  languages  like  ALGOL,  permit  more  concise  expression 
of  complex  program  steps. 
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J.   STRING  .OPERATIONS 

Many  RSTS/E  applications,  such  as  Computer  Assisted 
instruction,  text  editing,  and  business  data  processing, 
require  efficient  processing  of  alphabetic  data  such  as 
names,  addresses  and  even  entire  sentences.   BASIC-PLUS 
provides  for  the  processing  of  character  strings  of  various 
lengths,  the  maximum  length  being  limited  only  by  the  avail- 
able memory.   When  using  the  virtual  memory,  character 
strings  can  have  a  maximum  length  of  512  characters. 

A  comprehensive  group  of  string  operations  is  provided 
in  BASIC-PLUS.   Strings  may  be  appended  to   one  another. 
Strings  may  be  compared  to  one  another  to  see,  for  example, 
if  a  keyboard  response  is  correct  or  to  alphabetize  a  list 
of  names. 

Functions  are  available  to  extract,  examine  or  search 
for  a  string  of  characters  contained  within  a  larger  string. 
Further  enhancing  the  utility  of  string  variables  is  the 
capability  of  using  string  arrays  as  matrices.   With  this 
feature,  an  entire  list  of  alphabetic  data,  say  a  list  of 
names,  could  be  read-in  with  a  single  statement,  processed, 
and  output  with  another  statement.   In  standard  BASIC,  without 
string  arrays,  separate  READ  and  WRITE  statements  would  be 
required  for  each  name  in  the  list. 

K.   PROGRAMMABLE  TIMING  CONTROL 

BASIC-PLUS  gives  the  user  the  ability  to  control  certain 
operations  in  actual  time.   The  SLEEP  function  allows  the 
user  to  suspend  a  program  from  execution  for  a  specified 
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number  of  seconds.   When  this  time  interval  has  elapsed, 
execution  resumes.   Let  us  say  a  RSTS/E  installation  has  a 
substantial  number  of  users  trying  to  print  on  a  single 
line  printer.   Rather  than  each  one  of  these  users  getting 
in  a  queue,  inserting  a  SLEEP  command  in  his  program  to 
wait  a  few  seconds  if  the  line  printer  is  busy,  then  trying 
to  access  it  again,  consider  this  more  elegant  approach 
with  BASIC-PLUS.   Each  user  writes  his  line  printer  output 
into  a  specified  disk  file.   Then  a  program  running  at  the 
system  manager's  terminal  examines  the  disk  file  periodically 
and,  if  it  has  data  on  it,  prints  it  on  the  line  printer. 
If  the  disk  file  is  empty,  the  program  SLEEPS  a  few  seconds 
and  examines  it  again,  providing  optimum  throughput  without 
user  delay. 

In  some  applications,  the  length  of  time  a  terminal  user 
takes  to  respond  to  a  message  printed  at  his  terminal  is  a 
significant  variable.   The  WAIT  function  provides  an  interval 
timer  feature  which  may  be  used  for  signaling  the  program 
that  the  terminal  user  has  not  responded  within  some  speci- 
fied length  of  time.   One  example  of  the  use  of  the  WAIT 
function  is  in  CAI  applications  where  one  measure  of  student 
performance  may  be  "think  time."   If  the  student  takes  more 
than  five  seconds,  for  example,  to  respond  to  a  question, 
the  computer  can  restate  the  question  in  another  manner,  and 
record  the  delay  as  one  element  of  the  student's  overall 
performance. 

An  additional  real-time  feature  provides  year,  month, 
day  and  time-of-day  information  to  RSTS/E  programs. 
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L.   FORMATTED  OUTPUT 

Many  applications,  such  as  business  data  processing, 
require  more  flexible  control  of  the  printing  format  than 
Dartmouth  BASIC  allows.   BASIC-PLUS  includes  a  PRINT  USING 
statement  which   may  be  used  to  achieve  precise  definition 
of  printed  data  format.   PRINT  USING  allows  character, 
decimal  and  exponential  data  field  lengths  and  positions  to 
be  defined,  and  mixed,  in  a  line  of  output.   In  addition, 
leading  dollar  sign  or  asterisk  symbols  may  be  "floated" 
to  automatically  precede  the  most  significant  digit  of 
decimal  fields.   Also,  trailing  minus  signs  may  be  specified 
for  compatibility  with  accounting  report  standards. 

Format  BASIC-PLUS         Standard  BASIC 

Floating  dollar  sign  $95.20         $  95.2 

$4,382.69         $  4,382.69 
$0.43         $  0.43 
Asterisk  fill  $***20.32  not 

available 
$**120.48 
Comma 

insertion  4,832,684.15         4832684.15 

Decimal  point 

alignment  1,497.00  1497 

Trailing  minus 

sign  '  572.83-  -572.83 

M.   ERROR  RECOVERY 

One  of  the  more  frustrating  situations  for  a  timesharing 
terminal  user  is  having  a  program  cancelled  because  of  an 
input/output  error.   This  situation,  though  rare,  may  be 
eliminated  in  RSTS/E  by  use  of  the  ON  ERROR  GO  TO  statement. 
This  subroutine  call  statement  is  triggered  by  a  variety  of 
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input/output  operation  errors.   The  called  subroutine  is 
passed  a  value  which  identifies  the  error  type,  and  attempts 
to  recover  from  the  error  condition.   If  the  subroutine  is 
successful,  normal  execution  of  the  application  program 
resumes. 

Occasionally,  problems  will  occur  within  the  telephone 
system  causing  an  unexpected  disconnect  for  a  remote  user. 
In  this  event,  the  remote  terminal  may  be  cut  off  from  the 
job,  but  the  program  will  continue  to  execute.   The  user 
can  then  re-dial  the  computer  system,  re-attach  the  job, 
and  then  continue  interaction  with  the  program. 

In  all  cases,  on  hardware  or  software  error,  the  file 
system  is  kept  intact  and  secure.   In  the  unlikely  event  of 
a  system  "crash",  users  merely  have  to  perform  a  simple 
determination  of  the  status  of  their  file  processing  at 
the  time  of  the  crash,  and  then  continue. 

N.   EFFICIENT  SCHEDULING  ALGORITHM 

RSTS/E  installations  can  expect  exceptional  efficiency 
of  operation  because  the  operating  system  continuously  and 
dynamically  allocates  processor  time,  memory  space,  file 
space  and  peripheral  access  on  a  best-fit/best-throughput 
basis.   The  RSTS/E  operating  system  automatically  and  dynamically 
assigns  one  of  the  255  job  priority  levels  to  each  timesharing 
job.   These  priority  levels  are  based  on  such  criteria  as 
job  size,  computing  requirements,  current  time  since  last 
quantum  of  runtime  for  the  job,  and  input/output  requirements. 
They  may  also  be  altered  by  the  System  Manager. 
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Disk  allocation  is  made  dynamically  as  users  require. 
Users  do  not  have  to  plan  ahead  for  their  use  of  disk  space; 
however,  additional  efficiencies  may  be  realized  if  they  do. 
Specifying  contiguous  disk  segments  can  decrease  the  number 
of  disk  accesses  required  for  reading  and  writing  large 
files . 

0.   CONTROL  OF  USER  ACCESS  AND  RESOURCES 

RSTS/E  provides  facilities  to  aid  the  System  Manager  in 
accurate  and  efficient  control  of  system  use.   The  System 
Manager  may  specify  each  user's  programmer  and  project 
number,  password,  maximum  logged-out  disk  space  and  maximum 
number  of  files. 

If  desired,  user  access  to  the  system  can  be  controlled 
by  the  System  Manager.   In  fact,  if  desired,  access  could  be 
controlled  automatically,  through  a  program,  thus  relieving 
the  tedium  of  system  administration.   For  example,  in  a 
school,  certain  use  could  automatically  be  limited  to  30 
minutes  of  log-in  time  per  day  or  two  log-ins  per  day. 
Should  users  fail  to  log-off  at  the  designated  time,  the 
System  Manager  can  force  a  log-off  of  the  user's  terminal 
which  will  preserve  files,  but  terminate  job  execution. 

Facilities  are  available  for  the  System  Manager  to  send 
messages  to  all  terminal  users.   Also,  an  automatic  shutdown 
system  is  provided  which  periodically  warns  users  that  the 
system  will  shut  down  at  a  designated  time.   Any  users  still 
active  at  the  designated  time  are  logged-off  in  an  orderly 
fashion,  with  full  integrity  of  all  active  files. 
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Access  to  peripheral  devices  is  generally  open  to  all 
users  under  the  resource  sharing  concept  on  a  first-come, 
first-served  basis.  However,  the  capability  is  available 
to  the  System  Manager  to  intervene  in  peripheral  assignment. 
In  addition,  the  System  Manager  can  specify  how  the  space 
on  the  system  disks  is  to  be  allocated. 

P.   SYSTEM  USAGE  ACCOUNTING 

The  System  Manager,  as  well  as  any  terminal  user,  can 
determine  the  status  of  the  RSTS/E  system  through  use  of  the 
SYSTAT  program.   The  program  gives  information  of: 

1.  Status  of  all  jobs 

2.  Disk  structure  and  status 

3.  Status  of  other  peripheral  devices 

4.  Run-time  to  data 

A  more  detailed  accounting  of  specific  user,  of  all  users, 

is  possible  using  the  MONEY  program.   For  each  unique  account, 

MONEY  yields  information  on: 

1.  CPU  run-time 

2.  Connect  time  of  the  user's  terminal 

3 .  Memory  usage 

4.  Peripheral  device  usage 

5.  Number  of  log-ins  and  log-outs 

6.  Disk  storage  allocation 

Q.   SYSTEM  FILE  AND  SECURITY 

As  mentioned,  to  gain  access  to  a  RSTS/E  system,  a  user 
must  first  have  a  programmer  number  assigned  by  the  System 
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Manager.   Thereafter,  user  identity  is  established  by  entering 
number  and  password  (non-printing)  into  the  system.   Either 
the  user  or  System  Manager  has  the  capability  of  changing 
this  password  at  any  time.   This  facility,  when  combined 
with  the  individual  file  access  protection  codes,  provides 
an  effective  means  of  safeguarding  user  data. 

Additional  protection  can  be  provided  by  "private" 
removable  disk  packs  and  cartridges.   A  private  disk  is  one 
upon  which  only  authorized  users  may  create  files.   Other 
users  may  access  these  files  only  if  protection  codes  permit. 
Private  disks  may  be  mounted  or  dismounted  from  the  on-line 
system  at  any  time.   When  not  in  use,  they  may  be  kept  under 
lock  and  key. 

Each  terminal  user  has  full  control  over  the  degree  of 
privacy  desired  for  each  file  created.   Access  may  be 
limited  to  one  user,  to  those  in  the  same  group  (or  project), 
or  to  all  system  users.   Access  may  be  read-only,  write- 
only,  or  read/write. 

R.   BATCH  COBOL  OPTION 

A  RSTS/E  system  may  be  further  enhanced  for  business 
data  processing  applications  by  the  addition  of  the  PDP-11 
COBOL  language  processor.   COBOL  programs  and  run  in  batch 
mode  under  RSTS/E,  and  are  given  a  fixed  amount  of  execution 
time  under  the  scheduling  algorithm,  depending  on  the  number 
of  users  and  the  priority  level  assigned  to  the  jobs.   When 
a  batch  COBOL  job  is  executing,  response  at  BASIC-PLUS 
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terminals  is  not  appreciably  degraded,  since  the  COBOL  job 
competes  for  time  in  a  similar  fashion  as  all  other  users. 
COBOL  jobs  have  access  to  system  resources  in  the  same 
manner  as  BASIC-PLUS  jobs.   The  COBOL  language  processor 
conforms  to  the  ANSI  1974  standard. 

S.   COMMERCIAL  EXTENSIONS 

A  commercial  extension  package  is  available  to  enhance 
the  capabilities  of  the  RSTS/E  system  in  business  data 
processing  applications.   This  extension  package  consists 
of  a  disk  sort,  indexed  file  access  method,  decimal  arith- 
metic capacity  and  line  printer  spooling. 

1 .  Disk  Sort  Package 

The  disk  sort  package  is  a  series  of  programs 
allowing  the  user  to  sort  records  on  a  disk  file  into  a 
specified  order.   Up  to  15  different  fields  can  be  specified 
for  input  data  files  containing  up  to  32,650  records  —  up 
to  512  characters  per  record.   The  SORT  Program  may  be 
called  from  the  user  program  or  may  be  initiated  via  inter- 
active commands. 

2 .  Indexed  File  Access  Method 

The  Indexed  File  Access  Method  (IAM)  allows  the 
user  to  access  disk  file  data  records  randomly.   This  capa- 
bility allows  a  user  to  achieve  fast,  random  access  to 
data  records  without  concern  for  the  intricacies  of  disk 
file  organization.   Sequential  processing  of  these  records 
is  supported  either  directly  (if  there  have  been  no  records 
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added  to  the  file  since  the  last  file  organization)  or  by 
means  of  file  output  from  the  SORT  package. 

3 .  Decimal  Arithmetic  Option 

The  decimal  arithmetic  option  replaces  the  standard 
floating-point  arithmetic  with  four-word  fixed-point  arith- 
metic.  This  format  achieves  18  places  of  accuracy  with  12 
places  to  the  left  of  the  decimal  point.   Since  all  numbers 
represented  in  this  manner,  including  fractions,  are  true 
decimal  numbers,  there  can  be  no  cumulative  error  due  to 
repeated  operations.   For  this  reason,  the  representation 
is  normally  preferred  when  performing  accounting  functions. 

4 .  Line  Printer  Spooling 

The  line  printer  spooling  package  is  a  series  of 
BASIC-PLUS  programs  which  allow  the  user  to  specify  disk  or 
magnetic  tape  files  to  be  output  to  a  system  line  printer  — 
or  other  device.   To  utilize  the  spooler,  the  user  enters 
the  request  for  output;  the  request  is  queued  and  initiated 
when  the  output  device  becomes  available.   In  this  way, 
possible  conflicts  in  using  the  system  line  printer  are 
avoided.   User  programs  can  go  on  to  perform  other  tasks  nad 
system  throughput  can  often  be  increased  by  as  much  as  25 
percent.   Included  in  the  package  is  support  for  multiple 
line  printers. 
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